[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:32:04 dist-test-slave-2zvq krb5kdc[11306](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:32:04 dist-test-slave-2zvq krb5kdc[11306](info): set up 2 sockets
Sep 26 16:32:04 dist-test-slave-2zvq krb5kdc[11306](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260926 16:32:05.894116 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:32:05 dist-test-slave-2zvq krb5kdc[11306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:32:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:32:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
I20260926 16:32:06.014031 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:43407
--webserver_interface=127.11.8.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:40695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:43407
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:06.126022 11319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:06.126259 11319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:06.126377 11319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:06.130364 11319 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:32:06.130412 11319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:06.130431 11319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:06.130467 11319 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:32:06.130502 11319 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:32:06.135706 11319 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:40695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:43407
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:43407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:06.136678 11319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:06.137655 11319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:06.146101 11325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:06.146118 11327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:06.146101 11324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:06.146485 11319 server_base.cc:1034] running on GCE node
I20260926 16:32:06.147125 11319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:06.148208 11319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:06.149399 11319 hybrid_clock.cc:648] HybridClock initialized: now 1790440326149387 us; error 27 us; skew 500 ppm
Sep 26 16:32:06 dist-test-slave-2zvq krb5kdc[11306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440326, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:06.152139 11319 init.cc:377] Logged in from keytab as kudu/127.11.8.254@KRBTEST.COM (short username kudu)
I20260926 16:32:06.152988 11319 webserver.cc:460] Webserver started at http://127.11.8.254:37599/ using document root <none> and password file <none>
I20260926 16:32:06.153483 11319 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:06.153538 11319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:06.153694 11319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:06.155788 11319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "959c4431229a48eebfa886977b187618"
format_stamp: "Formatted at 2026-09-26 16:32:06 on dist-test-slave-2zvq"
server_key: "f2acc3f95cd1dba8bedd49f650f8b184"
server_key_iv: "027aaade977cbc164b09680f4b986fc6"
server_key_version: "encryptionkey@0"
I20260926 16:32:06.156433 11319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "959c4431229a48eebfa886977b187618"
format_stamp: "Formatted at 2026-09-26 16:32:06 on dist-test-slave-2zvq"
server_key: "f2acc3f95cd1dba8bedd49f650f8b184"
server_key_iv: "027aaade977cbc164b09680f4b986fc6"
server_key_version: "encryptionkey@0"
I20260926 16:32:06.160547 11319 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260926 16:32:06.162920 11334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:06.164230 11319 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:32:06.164366 11319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "959c4431229a48eebfa886977b187618"
format_stamp: "Formatted at 2026-09-26 16:32:06 on dist-test-slave-2zvq"
server_key: "f2acc3f95cd1dba8bedd49f650f8b184"
server_key_iv: "027aaade977cbc164b09680f4b986fc6"
server_key_version: "encryptionkey@0"
I20260926 16:32:06.164465 11319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:06.189132 11319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:06.192128 11319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:06.192346 11319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:06.198899 11319 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:43407
I20260926 16:32:06.198916 11386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:43407 every 8 connection(s)
I20260926 16:32:06.199913 11319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:32:06.202539 11387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:06.207895 11387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 959c4431229a48eebfa886977b187618: Bootstrap starting.
I20260926 16:32:06.210116 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 11319
I20260926 16:32:06.210234 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:32:06.210383 11387 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 959c4431229a48eebfa886977b187618: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:06.211076 11387 log.cc:824] T 00000000000000000000000000000000 P 959c4431229a48eebfa886977b187618: Log is configured to *not* fsync() on all Append() calls
I20260926 16:32:06.211795 11299 external_mini_cluster.cc:1428] Setting key d886e9d376fbf18294f763dc7ad29bae
I20260926 16:32:06.213085 11387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 959c4431229a48eebfa886977b187618: No bootstrap required, opened a new log
I20260926 16:32:06.217085 11387 raft_consensus.cc:348] T 00000000000000000000000000000000 P 959c4431229a48eebfa886977b187618 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "959c4431229a48eebfa886977b187618" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 43407 } }
I20260926 16:32:06.217325 11387 raft_consensus.cc:374] T 00000000000000000000000000000000 P 959c4431229a48eebfa886977b187618 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:06.217403 11387 raft_consensus.cc:729] T 00000000000000000000000000000000 P 959c4431229a48eebfa886977b187618 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 959c4431229a48eebfa886977b187618, State: Initialized, Role: FOLLOWER
I20260926 16:32:06.217913 11387 consensus_queue.cc:260] T 00000000000000000000000000000000 P 959c4431229a48eebfa886977b187618 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "959c4431229a48eebfa886977b187618" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 43407 } }
I20260926 16:32:06.218067 11387 raft_consensus.cc:388] T 00000000000000000000000000000000 P 959c4431229a48eebfa886977b187618 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:32:06.218150 11387 raft_consensus.cc:482] T 00000000000000000000000000000000 P 959c4431229a48eebfa886977b187618 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:32:06.218255 11387 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 959c4431229a48eebfa886977b187618 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:06.219375 11387 raft_consensus.cc:504] T 00000000000000000000000000000000 P 959c4431229a48eebfa886977b187618 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "959c4431229a48eebfa886977b187618" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 43407 } }
I20260926 16:32:06.219717 11387 leader_election.cc:302] T 00000000000000000000000000000000 P 959c4431229a48eebfa886977b187618 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 959c4431229a48eebfa886977b187618; no voters:
I20260926 16:32:06.220024 11387 leader_election.cc:288] T 00000000000000000000000000000000 P 959c4431229a48eebfa886977b187618 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:32:06.220183 11392 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 959c4431229a48eebfa886977b187618 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:32:06.220435 11392 raft_consensus.cc:686] T 00000000000000000000000000000000 P 959c4431229a48eebfa886977b187618 [term 1 LEADER]: Becoming Leader. State: Replica: 959c4431229a48eebfa886977b187618, State: Running, Role: LEADER
I20260926 16:32:06.220733 11392 consensus_queue.cc:237] T 00000000000000000000000000000000 P 959c4431229a48eebfa886977b187618 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "959c4431229a48eebfa886977b187618" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 43407 } }
I20260926 16:32:06.221150 11387 sys_catalog.cc:564] T 00000000000000000000000000000000 P 959c4431229a48eebfa886977b187618 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:32:06.222412 11393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 959c4431229a48eebfa886977b187618 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "959c4431229a48eebfa886977b187618" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "959c4431229a48eebfa886977b187618" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 43407 } } }
I20260926 16:32:06.222541 11393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 959c4431229a48eebfa886977b187618 [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:06.222774 11393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 959c4431229a48eebfa886977b187618 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 959c4431229a48eebfa886977b187618. Latest consensus state: current_term: 1 leader_uuid: "959c4431229a48eebfa886977b187618" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "959c4431229a48eebfa886977b187618" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 43407 } } }
I20260926 16:32:06.222877 11393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 959c4431229a48eebfa886977b187618 [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:06.223145 11403 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:32:06.226294 11403 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 26 16:32:06 dist-test-slave-2zvq krb5kdc[11306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:32:06.231809 11403 catalog_manager.cc:1347] Generated new cluster ID: 9b3d0a52ccd94e7093f55e63549925e0
I20260926 16:32:06.231931 11403 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:32:06.237325 11390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:06.214561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58472 (local address 127.11.8.254:43407)
0926 16:32:06.215072 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:06.215093 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:06.216180 (+ 1087us) server_negotiation.cc:407] Connection header received
0926 16:32:06.219033 (+ 2853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:06.219054 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:06.219450 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:06.219814 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:06.222971 (+ 3157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:06.224468 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:06.225804 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:06.226475 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:06.226639 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:06.231246 (+ 4607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:06.231252 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:06.231267 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:06.231297 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:06.233752 (+ 2455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:06.234300 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:06.234307 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:06.234313 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:06.234432 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:06.234801 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:06.234805 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:06.234806 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:06.235196 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:06.235446 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:06.235977 (+ 531us) server_negotiation.cc:299] Negotiation successful
0926 16:32:06.236241 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":124,"threads_started":1}
I20260926 16:32:06.248296 11403 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:32:06.249153 11403 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:32:06.256048 11403 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 959c4431229a48eebfa886977b187618: Generated new TSK 0
I20260926 16:32:06.256770 11403 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.8.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab.
Entry for principal kudu/127.11.8.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab.
I20260926 16:32:06.285666 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.193:0
--local_ip_for_outbound_sockets=127.11.8.193
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:43407
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:40695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:06.430390 11413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:06.430671 11413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:06.430743 11413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:06.434866 11413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:06.434945 11413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:06.435037 11413 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.193
I20260926 16:32:06.440169 11413 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:40695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:43407
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11413
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.193
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:06.441247 11413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:06.442349 11413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:06.451479 11419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:06.451570 11413 server_base.cc:1034] running on GCE node
W20260926 16:32:06.451545 11418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:06.451495 11421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:06.452026 11413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:06.452684 11413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:06.453866 11413 hybrid_clock.cc:648] HybridClock initialized: now 1790440326453850 us; error 30 us; skew 500 ppm
Sep 26 16:32:06 dist-test-slave-2zvq krb5kdc[11306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440326, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:06.456696 11413 init.cc:377] Logged in from keytab as kudu/127.11.8.193@KRBTEST.COM (short username kudu)
I20260926 16:32:06.457491 11413 webserver.cc:460] Webserver started at http://127.11.8.193:36545/ using document root <none> and password file <none>
I20260926 16:32:06.458042 11413 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:06.458132 11413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:06.458398 11413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:06.460496 11413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-0/data/instance:
uuid: "05b7e5197faa4bbf8d9642f8e7f2755f"
format_stamp: "Formatted at 2026-09-26 16:32:06 on dist-test-slave-2zvq"
server_key: "a8354cc8a69b2d82d0aa082014ec0319"
server_key_iv: "46df24d182bd6db7d99067cfb0bf1213"
server_key_version: "encryptionkey@0"
I20260926 16:32:06.461139 11413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance:
uuid: "05b7e5197faa4bbf8d9642f8e7f2755f"
format_stamp: "Formatted at 2026-09-26 16:32:06 on dist-test-slave-2zvq"
server_key: "a8354cc8a69b2d82d0aa082014ec0319"
server_key_iv: "46df24d182bd6db7d99067cfb0bf1213"
server_key_version: "encryptionkey@0"
I20260926 16:32:06.465257 11413 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260926 16:32:06.467592 11428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:06.468783 11413 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 16:32:06.468905 11413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-0/wal
uuid: "05b7e5197faa4bbf8d9642f8e7f2755f"
format_stamp: "Formatted at 2026-09-26 16:32:06 on dist-test-slave-2zvq"
server_key: "a8354cc8a69b2d82d0aa082014ec0319"
server_key_iv: "46df24d182bd6db7d99067cfb0bf1213"
server_key_version: "encryptionkey@0"
I20260926 16:32:06.469026 11413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:06.490819 11413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:06.494354 11413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:06.494566 11413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:06.495162 11413 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:32:06.496001 11413 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:06.496070 11413 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:06.496140 11413 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:06.496181 11413 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:06.505107 11413 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.193:38623
I20260926 16:32:06.505133 11541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.193:38623 every 8 connection(s)
I20260926 16:32:06.506037 11413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
I20260926 16:32:06.513280 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 11413
I20260926 16:32:06.513401 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance
I20260926 16:32:06.513630 11299 external_mini_cluster.cc:1428] Setting key 821f66e28cb107a8fa80220a3ec62933
Sep 26 16:32:06 dist-test-slave-2zvq krb5kdc[11306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440326, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:32:06.520462 11390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:06.507891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:55417 (local address 127.11.8.254:43407)
0926 16:32:06.508059 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:06.508063 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:06.509153 (+ 1090us) server_negotiation.cc:407] Connection header received
0926 16:32:06.510074 (+ 921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:06.510077 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:06.510134 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:06.510217 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:06.511385 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:06.511924 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:06.513078 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:06.513313 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:06.513433 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:06.516525 (+ 3092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:06.516531 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:06.516537 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:06.516567 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:06.518235 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:06.518885 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:06.518892 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:06.518897 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:06.518991 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:06.519304 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:06.519310 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:06.519315 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:06.519572 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:06.519677 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:06.520124 (+ 447us) server_negotiation.cc:299] Negotiation successful
0926 16:32:06.520262 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 16:32:06.521183 11544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:06.508279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:43407 (local address 127.11.8.193:55417)
0926 16:32:06.508978 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:06.509021 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:06.509860 (+ 839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:06.510405 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:06.510412 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:06.510760 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:06.511232 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:06.511244 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:06.512049 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:06.512055 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:06.512932 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:06.512941 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:06.513469 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:06.513473 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:06.513641 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:06.514493 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:06.514508 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:06.516331 (+ 1823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:06.518386 (+ 2055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:06.518394 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:06.518412 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:06.518780 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:06.519084 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:06.519087 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:06.519088 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:06.519213 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:06.519672 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:06.519677 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:06.519890 (+ 213us) client_negotiation.cc:769] Sending connection context
0926 16:32:06.520047 (+ 157us) client_negotiation.cc:240] Negotiation successful
0926 16:32:06.520284 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":117,"threads_started":1}
I20260926 16:32:06.522596 11542 heartbeater.cc:347] Connected to a master server at 127.11.8.254:43407
I20260926 16:32:06.522861 11542 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:06.523450 11542 heartbeater.cc:511] Master 127.11.8.254:43407 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.8.194@KRBTEST.COM; defaulting to no policy
I20260926 16:32:06.525028 11351 ts_manager.cc:194] Registered new tserver with Master: 05b7e5197faa4bbf8d9642f8e7f2755f (127.11.8.193:38623)
I20260926 16:32:06.526180 11351 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.193@KRBTEST.COM'} at 127.11.8.193:55417
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab.
Entry for principal kudu/127.11.8.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab.
I20260926 16:32:06.541661 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.194:0
--local_ip_for_outbound_sockets=127.11.8.194
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:43407
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:40695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:06.649276 11547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:06.649515 11547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:06.649557 11547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:06.653425 11547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:06.653475 11547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:06.653532 11547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.194
I20260926 16:32:06.658535 11547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:40695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:43407
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.194
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:06.659433 11547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:06.660342 11547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:06.669497 11552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:06.669533 11553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:06.669533 11555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:06.669729 11547 server_base.cc:1034] running on GCE node
I20260926 16:32:06.670104 11547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:06.670799 11547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:06.671980 11547 hybrid_clock.cc:648] HybridClock initialized: now 1790440326671963 us; error 30 us; skew 500 ppm
Sep 26 16:32:06 dist-test-slave-2zvq krb5kdc[11306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440326, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:06.674870 11547 init.cc:377] Logged in from keytab as kudu/127.11.8.194@KRBTEST.COM (short username kudu)
I20260926 16:32:06.675658 11547 webserver.cc:460] Webserver started at http://127.11.8.194:34089/ using document root <none> and password file <none>
I20260926 16:32:06.676194 11547 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:06.676280 11547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:06.676515 11547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:06.678673 11547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-1/data/instance:
uuid: "c41ba1e7a642476aa65c5b7d50a1b7d0"
format_stamp: "Formatted at 2026-09-26 16:32:06 on dist-test-slave-2zvq"
server_key: "6ddc1891a8e9b305df7ae109fd7b53eb"
server_key_iv: "2d25ba858cf0067d415ab3cc1a8e6d31"
server_key_version: "encryptionkey@0"
I20260926 16:32:06.679340 11547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance:
uuid: "c41ba1e7a642476aa65c5b7d50a1b7d0"
format_stamp: "Formatted at 2026-09-26 16:32:06 on dist-test-slave-2zvq"
server_key: "6ddc1891a8e9b305df7ae109fd7b53eb"
server_key_iv: "2d25ba858cf0067d415ab3cc1a8e6d31"
server_key_version: "encryptionkey@0"
I20260926 16:32:06.683235 11547 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260926 16:32:06.685474 11562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:06.686648 11547 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260926 16:32:06.686776 11547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-1/wal
uuid: "c41ba1e7a642476aa65c5b7d50a1b7d0"
format_stamp: "Formatted at 2026-09-26 16:32:06 on dist-test-slave-2zvq"
server_key: "6ddc1891a8e9b305df7ae109fd7b53eb"
server_key_iv: "2d25ba858cf0067d415ab3cc1a8e6d31"
server_key_version: "encryptionkey@0"
I20260926 16:32:06.686882 11547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:06.712148 11547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:06.715198 11547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:06.715405 11547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:06.715977 11547 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:32:06.716735 11547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:06.716804 11547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:06.716881 11547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:06.716930 11547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:06.726327 11547 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.194:36835
I20260926 16:32:06.726361 11675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.194:36835 every 8 connection(s)
I20260926 16:32:06.727216 11547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
Sep 26 16:32:06 dist-test-slave-2zvq krb5kdc[11306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440326, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:32:06.737316 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 11547
I20260926 16:32:06.737404 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance
I20260926 16:32:06.737744 11299 external_mini_cluster.cc:1428] Setting key 47f632bb82c3992ff550cb23d75179c1
I20260926 16:32:06.739732 11390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:06.728983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:50785 (local address 127.11.8.254:43407)
0926 16:32:06.729110 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:06.729113 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:06.730045 (+ 932us) server_negotiation.cc:407] Connection header received
0926 16:32:06.730921 (+ 876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:06.730924 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:06.730968 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:06.731039 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:06.732095 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:06.732617 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:06.733687 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:06.733900 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:06.733975 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:06.736209 (+ 2234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:06.736212 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:06.736214 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:06.736237 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:06.737503 (+ 1266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:06.738046 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:06.738049 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:06.738050 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:06.738125 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:06.738563 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:06.738567 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:06.738568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:06.738784 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:06.738907 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:06.739440 (+ 533us) server_negotiation.cc:299] Negotiation successful
0926 16:32:06.739557 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 16:32:06.741237 11678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:06.729286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:43407 (local address 127.11.8.194:50785)
0926 16:32:06.729907 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:06.729946 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:06.730719 (+ 773us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:06.731189 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:06.731195 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:06.731527 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:06.731962 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:06.731975 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:06.732723 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:06.732726 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:06.733583 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:06.733592 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:06.734031 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:06.734033 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:06.734165 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:06.734710 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:06.734725 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:06.736064 (+ 1339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:06.737600 (+ 1536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:06.737606 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:06.737621 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:06.737952 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:06.738227 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:06.738229 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:06.738230 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:06.738445 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:06.738882 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:06.738887 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:06.739158 (+ 271us) client_negotiation.cc:769] Sending connection context
0926 16:32:06.739888 (+ 730us) client_negotiation.cc:240] Negotiation successful
0926 16:32:06.740228 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":104,"threads_started":1}
I20260926 16:32:06.742425 11676 heartbeater.cc:347] Connected to a master server at 127.11.8.254:43407
I20260926 16:32:06.742650 11676 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:06.743099 11676 heartbeater.cc:511] Master 127.11.8.254:43407 requested a full tablet report, sending...
I20260926 16:32:06.744107 11351 ts_manager.cc:194] Registered new tserver with Master: c41ba1e7a642476aa65c5b7d50a1b7d0 (127.11.8.194:36835)
I20260926 16:32:06.744668 11351 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.194@KRBTEST.COM'} at 127.11.8.194:50785
WARNING: no policy specified for kudu/127.11.8.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab.
Entry for principal kudu/127.11.8.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab.
I20260926 16:32:06.763535 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.195:0
--local_ip_for_outbound_sockets=127.11.8.195
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:43407
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:40695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:06.874621 11681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:06.874866 11681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:06.874909 11681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:06.878836 11681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:06.878892 11681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:06.878950 11681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.195
I20260926 16:32:06.884063 11681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:40695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:43407
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.195
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:06.885046 11681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:06.885998 11681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:06.895182 11689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:06.895166 11687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:06.895188 11686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:06.895463 11681 server_base.cc:1034] running on GCE node
I20260926 16:32:06.895910 11681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:06.896520 11681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:06.897756 11681 hybrid_clock.cc:648] HybridClock initialized: now 1790440326897737 us; error 29 us; skew 500 ppm
Sep 26 16:32:06 dist-test-slave-2zvq krb5kdc[11306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440326, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:06.900533 11681 init.cc:377] Logged in from keytab as kudu/127.11.8.195@KRBTEST.COM (short username kudu)
I20260926 16:32:06.901378 11681 webserver.cc:460] Webserver started at http://127.11.8.195:43709/ using document root <none> and password file <none>
I20260926 16:32:06.901925 11681 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:06.902014 11681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:06.902238 11681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:06.904371 11681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-2/data/instance:
uuid: "7bd23971194640b9a99624d170b27ca2"
format_stamp: "Formatted at 2026-09-26 16:32:06 on dist-test-slave-2zvq"
server_key: "43a9b7f6215e0c93264dbb461366d4af"
server_key_iv: "f0a30b2ed5281c00016ae9bd2eae3138"
server_key_version: "encryptionkey@0"
I20260926 16:32:06.905056 11681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance:
uuid: "7bd23971194640b9a99624d170b27ca2"
format_stamp: "Formatted at 2026-09-26 16:32:06 on dist-test-slave-2zvq"
server_key: "43a9b7f6215e0c93264dbb461366d4af"
server_key_iv: "f0a30b2ed5281c00016ae9bd2eae3138"
server_key_version: "encryptionkey@0"
I20260926 16:32:06.908941 11681 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260926 16:32:06.911298 11696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:06.912451 11681 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:32:06.912590 11681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-2/wal
uuid: "7bd23971194640b9a99624d170b27ca2"
format_stamp: "Formatted at 2026-09-26 16:32:06 on dist-test-slave-2zvq"
server_key: "43a9b7f6215e0c93264dbb461366d4af"
server_key_iv: "f0a30b2ed5281c00016ae9bd2eae3138"
server_key_version: "encryptionkey@0"
I20260926 16:32:06.912700 11681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:06.922570 11681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:06.925350 11681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:06.925546 11681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:06.926079 11681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:32:06.926905 11681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:06.926973 11681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:06.927042 11681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:06.927089 11681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:06.936025 11681 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.195:44795
I20260926 16:32:06.936056 11809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.195:44795 every 8 connection(s)
I20260926 16:32:06.936954 11681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
I20260926 16:32:06.939131 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 11681
I20260926 16:32:06.939252 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance
I20260926 16:32:06.939563 11299 external_mini_cluster.cc:1428] Setting key 69839ddc0b7426b90c67916c394cfe85
Sep 26 16:32:06 dist-test-slave-2zvq krb5kdc[11306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440326, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:32:06.951789 11390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:06.938829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:57729 (local address 127.11.8.254:43407)
0926 16:32:06.938993 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:06.938996 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:06.940082 (+ 1086us) server_negotiation.cc:407] Connection header received
0926 16:32:06.941119 (+ 1037us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:06.941125 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:06.941175 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:06.941250 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:06.942623 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:06.943186 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:06.944682 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:06.944998 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:06.945119 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:06.948525 (+ 3406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:06.948528 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:06.948530 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:06.948553 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:06.950000 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:06.950541 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:06.950543 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:06.950544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:06.950618 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:06.950933 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:06.950934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:06.950935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:06.951084 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:06.951153 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:06.951584 (+ 431us) server_negotiation.cc:299] Negotiation successful
0926 16:32:06.951675 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260926 16:32:06.952623 11812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:06.939195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:43407 (local address 127.11.8.195:57729)
0926 16:32:06.939921 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:06.939962 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:06.940902 (+ 940us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:06.941435 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:06.941443 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:06.941949 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:06.942493 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:06.942508 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:06.943321 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:06.943324 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:06.944540 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:06.944554 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:06.945158 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:06.945161 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:06.945414 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:06.946847 (+ 1433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:06.946869 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:06.948382 (+ 1513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:06.950112 (+ 1730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:06.950117 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:06.950128 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:06.950455 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:06.950696 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:06.950698 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:06.950699 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:06.950856 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:06.951165 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:06.951170 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:06.951380 (+ 210us) client_negotiation.cc:769] Sending connection context
0926 16:32:06.951520 (+ 140us) client_negotiation.cc:240] Negotiation successful
0926 16:32:06.951746 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":138,"threads_started":1}
I20260926 16:32:06.953737 11810 heartbeater.cc:347] Connected to a master server at 127.11.8.254:43407
I20260926 16:32:06.953979 11810 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:06.954491 11810 heartbeater.cc:511] Master 127.11.8.254:43407 requested a full tablet report, sending...
I20260926 16:32:06.955463 11351 ts_manager.cc:194] Registered new tserver with Master: 7bd23971194640b9a99624d170b27ca2 (127.11.8.195:44795)
I20260926 16:32:06.956024 11351 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.195@KRBTEST.COM'} at 127.11.8.195:57729
I20260926 16:32:06.965428 11299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 16:32:06 dist-test-slave-2zvq krb5kdc[11306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440326, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 16:32:06 dist-test-slave-2zvq krb5kdc[11306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440326, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.8.193@KRBTEST.COM
I20260926 16:32:06.990429 11817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:06.981215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55520 (local address 127.11.8.193:38623)
0926 16:32:06.981480 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:06.981498 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:06.981590 (+ 92us) server_negotiation.cc:407] Connection header received
0926 16:32:06.981749 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:06.981756 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:06.981960 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:06.982109 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:06.982811 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:06.983573 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:06.984409 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:06.984757 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:06.984879 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:06.987176 (+ 2297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:06.987185 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:06.987195 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:06.987218 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:06.988615 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:06.989046 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:06.989053 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:06.989057 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:06.989138 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:06.989409 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:06.989414 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:06.989416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:06.989682 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:06.989899 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:06.990069 (+ 170us) server_negotiation.cc:299] Negotiation successful
0926 16:32:06.990218 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":74,"threads_started":1}
W20260926 16:32:06.991401 11456 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55520
Sep 26 16:32:06 dist-test-slave-2zvq krb5kdc[11306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 16:32:07 dist-test-slave-2zvq krb5kdc[11306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.193@KRBTEST.COM
I20260926 16:32:07.015182 11817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:07.007671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55534 (local address 127.11.8.193:38623)
0926 16:32:07.007855 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:07.007858 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:07.007906 (+ 48us) server_negotiation.cc:407] Connection header received
0926 16:32:07.008021 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:07.008023 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:07.008062 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:07.008129 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:07.008772 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:07.009259 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:07.010054 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:07.010317 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:07.010400 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:07.012638 (+ 2238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:07.012641 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:07.012643 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:07.012663 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:07.013858 (+ 1195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:07.014211 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:07.014213 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:07.014215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:07.014330 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:07.014567 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:07.014569 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:07.014571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:07.014710 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:07.014805 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:07.014979 (+ 174us) server_negotiation.cc:299] Negotiation successful
0926 16:32:07.015067 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260926 16:32:07.016147 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 11413
I20260926 16:32:07.022096 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 11547
I20260926 16:32:07.029175 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 11681
I20260926 16:32:07.034365 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 11319
2026-09-26T16:32:07Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2206 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:32:07 dist-test-slave-2zvq krb5kdc[11824](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:32:07 dist-test-slave-2zvq krb5kdc[11824](info): set up 2 sockets
Sep 26 16:32:07 dist-test-slave-2zvq krb5kdc[11824](info): commencing operation
krb5kdc: starting...
W20260926 16:32:08.063576 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:32:08 dist-test-slave-2zvq krb5kdc[11824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:32:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:32:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
I20260926 16:32:08.183424 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:35643
--webserver_interface=127.11.8.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:36989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:35643
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:08.297681 11837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:08.297968 11837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:08.298040 11837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:08.301997 11837 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:32:08.302086 11837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:08.302124 11837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:08.302179 11837 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:32:08.302219 11837 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:32:08.307320 11837 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:36989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:35643
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:35643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:08.308305 11837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:08.309281 11837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:08.317304 11845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:08.317309 11843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:08.317312 11842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:08.317565 11837 server_base.cc:1034] running on GCE node
I20260926 16:32:08.318141 11837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:08.319212 11837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:08.320371 11837 hybrid_clock.cc:648] HybridClock initialized: now 1790440328320352 us; error 30 us; skew 500 ppm
Sep 26 16:32:08 dist-test-slave-2zvq krb5kdc[11824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440328, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:08.323093 11837 init.cc:377] Logged in from keytab as kudu/127.11.8.254@KRBTEST.COM (short username kudu)
I20260926 16:32:08.323968 11837 webserver.cc:460] Webserver started at http://127.11.8.254:44621/ using document root <none> and password file <none>
I20260926 16:32:08.324517 11837 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:08.324606 11837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:08.324831 11837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:08.327005 11837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "ff98ac08b7a24119a6463801c13f1a07"
format_stamp: "Formatted at 2026-09-26 16:32:08 on dist-test-slave-2zvq"
server_key: "57c82cc32fb7a5e93bd11bb64239a969"
server_key_iv: "f4938ad2d94a7495fd94c118188f008a"
server_key_version: "encryptionkey@0"
I20260926 16:32:08.327632 11837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "ff98ac08b7a24119a6463801c13f1a07"
format_stamp: "Formatted at 2026-09-26 16:32:08 on dist-test-slave-2zvq"
server_key: "57c82cc32fb7a5e93bd11bb64239a969"
server_key_iv: "f4938ad2d94a7495fd94c118188f008a"
server_key_version: "encryptionkey@0"
I20260926 16:32:08.331319 11837 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260926 16:32:08.333595 11852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:08.334775 11837 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:32:08.334910 11837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "ff98ac08b7a24119a6463801c13f1a07"
format_stamp: "Formatted at 2026-09-26 16:32:08 on dist-test-slave-2zvq"
server_key: "57c82cc32fb7a5e93bd11bb64239a969"
server_key_iv: "f4938ad2d94a7495fd94c118188f008a"
server_key_version: "encryptionkey@0"
I20260926 16:32:08.335021 11837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:08.363986 11837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:08.366830 11837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:08.367035 11837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:08.374032 11837 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:35643
I20260926 16:32:08.374058 11904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:35643 every 8 connection(s)
I20260926 16:32:08.374934 11837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:32:08.377468 11905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:08.378767 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 11837
I20260926 16:32:08.378895 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:32:08.379174 11299 external_mini_cluster.cc:1428] Setting key 7de206e9059d8fc311fb319c68138343
I20260926 16:32:08.383411 11905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff98ac08b7a24119a6463801c13f1a07: Bootstrap starting.
I20260926 16:32:08.385557 11905 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff98ac08b7a24119a6463801c13f1a07: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:08.386451 11905 log.cc:824] T 00000000000000000000000000000000 P ff98ac08b7a24119a6463801c13f1a07: Log is configured to *not* fsync() on all Append() calls
Sep 26 16:32:08 dist-test-slave-2zvq krb5kdc[11824](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:32:08.388731 11905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff98ac08b7a24119a6463801c13f1a07: No bootstrap required, opened a new log
I20260926 16:32:08.392796 11905 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff98ac08b7a24119a6463801c13f1a07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff98ac08b7a24119a6463801c13f1a07" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 35643 } }
I20260926 16:32:08.393007 11905 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff98ac08b7a24119a6463801c13f1a07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:08.393065 11905 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff98ac08b7a24119a6463801c13f1a07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff98ac08b7a24119a6463801c13f1a07, State: Initialized, Role: FOLLOWER
I20260926 16:32:08.393081 11908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:08.380526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38112 (local address 127.11.8.254:35643)
0926 16:32:08.380943 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:08.380957 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:08.380991 (+ 34us) server_negotiation.cc:407] Connection header received
0926 16:32:08.381593 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:08.381609 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:08.381929 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:08.382260 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:08.383086 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:08.384102 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:08.384976 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:08.385435 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:08.385604 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:08.388514 (+ 2910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:08.388519 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:08.388533 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:08.388557 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:08.390216 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:08.390812 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:08.390817 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:08.390821 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:08.390947 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:08.391230 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:08.391232 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:08.391233 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:08.391583 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:08.391732 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:08.392018 (+ 286us) server_negotiation.cc:299] Negotiation successful
0926 16:32:08.392312 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":130,"threads_started":1}
I20260926 16:32:08.393489 11905 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff98ac08b7a24119a6463801c13f1a07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff98ac08b7a24119a6463801c13f1a07" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 35643 } }
I20260926 16:32:08.393641 11905 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff98ac08b7a24119a6463801c13f1a07 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:32:08.393728 11905 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff98ac08b7a24119a6463801c13f1a07 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:32:08.393847 11905 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff98ac08b7a24119a6463801c13f1a07 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:08.394924 11905 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff98ac08b7a24119a6463801c13f1a07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff98ac08b7a24119a6463801c13f1a07" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 35643 } }
I20260926 16:32:08.395260 11905 leader_election.cc:302] T 00000000000000000000000000000000 P ff98ac08b7a24119a6463801c13f1a07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff98ac08b7a24119a6463801c13f1a07; no voters:
I20260926 16:32:08.395622 11905 leader_election.cc:288] T 00000000000000000000000000000000 P ff98ac08b7a24119a6463801c13f1a07 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:32:08.395687 11910 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff98ac08b7a24119a6463801c13f1a07 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:32:08.395877 11910 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff98ac08b7a24119a6463801c13f1a07 [term 1 LEADER]: Becoming Leader. State: Replica: ff98ac08b7a24119a6463801c13f1a07, State: Running, Role: LEADER
I20260926 16:32:08.396160 11910 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff98ac08b7a24119a6463801c13f1a07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff98ac08b7a24119a6463801c13f1a07" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 35643 } }
I20260926 16:32:08.396813 11905 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff98ac08b7a24119a6463801c13f1a07 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:32:08.397833 11912 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff98ac08b7a24119a6463801c13f1a07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff98ac08b7a24119a6463801c13f1a07. Latest consensus state: current_term: 1 leader_uuid: "ff98ac08b7a24119a6463801c13f1a07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff98ac08b7a24119a6463801c13f1a07" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 35643 } } }
I20260926 16:32:08.397846 11911 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff98ac08b7a24119a6463801c13f1a07 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff98ac08b7a24119a6463801c13f1a07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff98ac08b7a24119a6463801c13f1a07" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 35643 } } }
I20260926 16:32:08.397984 11911 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff98ac08b7a24119a6463801c13f1a07 [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:08.397982 11912 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff98ac08b7a24119a6463801c13f1a07 [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:08.398545 11919 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:32:08.401137 11919 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:32:08.409103 11919 catalog_manager.cc:1347] Generated new cluster ID: 6e29c299d5a544129f15e5d7396d5d78
I20260926 16:32:08.409206 11919 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:32:08.433799 11919 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:32:08.435815 11919 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:32:08.450918 11919 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff98ac08b7a24119a6463801c13f1a07: Generated new TSK 0
I20260926 16:32:08.451924 11919 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.8.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab.
Entry for principal kudu/127.11.8.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab.
I20260926 16:32:08.479076 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.193:0
--local_ip_for_outbound_sockets=127.11.8.193
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:35643
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:36989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:08.588202 11931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:08.588483 11931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:08.588559 11931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:08.592618 11931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:08.592702 11931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:08.592799 11931 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.193
I20260926 16:32:08.598583 11931 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:36989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:35643
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.193
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:08.599677 11931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:08.600728 11931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:08.610392 11936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:08.610399 11939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:08.610399 11937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:08.610689 11931 server_base.cc:1034] running on GCE node
I20260926 16:32:08.611053 11931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:08.611685 11931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:08.612886 11931 hybrid_clock.cc:648] HybridClock initialized: now 1790440328612865 us; error 33 us; skew 500 ppm
Sep 26 16:32:08 dist-test-slave-2zvq krb5kdc[11824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440328, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:08.615613 11931 init.cc:377] Logged in from keytab as kudu/127.11.8.193@KRBTEST.COM (short username kudu)
I20260926 16:32:08.616469 11931 webserver.cc:460] Webserver started at http://127.11.8.193:34291/ using document root <none> and password file <none>
I20260926 16:32:08.617033 11931 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:08.617120 11931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:08.617388 11931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:08.619791 11931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-0/data/instance:
uuid: "9eda3ec510694f8c966192e03505c1d0"
format_stamp: "Formatted at 2026-09-26 16:32:08 on dist-test-slave-2zvq"
server_key: "5b7fcda16ab69a036fd8492265cdefd9"
server_key_iv: "b21d11231d07e7ccab30c5f20b335c34"
server_key_version: "encryptionkey@0"
I20260926 16:32:08.620427 11931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance:
uuid: "9eda3ec510694f8c966192e03505c1d0"
format_stamp: "Formatted at 2026-09-26 16:32:08 on dist-test-slave-2zvq"
server_key: "5b7fcda16ab69a036fd8492265cdefd9"
server_key_iv: "b21d11231d07e7ccab30c5f20b335c34"
server_key_version: "encryptionkey@0"
I20260926 16:32:08.624224 11931 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260926 16:32:08.626633 11946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:08.627818 11931 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:32:08.627965 11931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-0/wal
uuid: "9eda3ec510694f8c966192e03505c1d0"
format_stamp: "Formatted at 2026-09-26 16:32:08 on dist-test-slave-2zvq"
server_key: "5b7fcda16ab69a036fd8492265cdefd9"
server_key_iv: "b21d11231d07e7ccab30c5f20b335c34"
server_key_version: "encryptionkey@0"
I20260926 16:32:08.628064 11931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:08.650569 11931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:08.653446 11931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:08.653640 11931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:08.654207 11931 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:32:08.655033 11931 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:08.655077 11931 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:08.655117 11931 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:08.655133 11931 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:08.663970 11931 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.193:34459
I20260926 16:32:08.664008 12059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.193:34459 every 8 connection(s)
I20260926 16:32:08.664860 11931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
I20260926 16:32:08.665140 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 11931
I20260926 16:32:08.665264 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance
I20260926 16:32:08.665539 11299 external_mini_cluster.cc:1428] Setting key 7155e78b409cb02945f263084fe7c5f3
Sep 26 16:32:08 dist-test-slave-2zvq krb5kdc[11824](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440328, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.8.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.194@KRBTEST.COM" created.
I20260926 16:32:08.680234 11908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:08.666725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:44717 (local address 127.11.8.254:35643)
0926 16:32:08.666857 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:08.666861 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:08.667835 (+ 974us) server_negotiation.cc:407] Connection header received
0926 16:32:08.669499 (+ 1664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:08.669502 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:08.669589 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:08.669705 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:08.671073 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:08.671607 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:08.673439 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:08.673663 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:08.673744 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:08.676567 (+ 2823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:08.676570 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:08.676572 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:08.676593 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:08.677932 (+ 1339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:08.678560 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:08.678564 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:08.678567 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:08.678689 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:08.679112 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:08.679118 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:08.679123 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:08.679330 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:08.679446 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:08.679947 (+ 501us) server_negotiation.cc:299] Negotiation successful
0926 16:32:08.680066 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 16:32:08.680959 12062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:08.667175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:35643 (local address 127.11.8.193:44717)
0926 16:32:08.667656 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:08.667714 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:08.669224 (+ 1510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:08.669990 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:08.669998 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:08.670468 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:08.670923 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:08.670937 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:08.671740 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:08.671746 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:08.673310 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:08.673325 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:08.673821 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:08.673828 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:08.674064 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:08.674662 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:08.674686 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:08.676380 (+ 1694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:08.678077 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:08.678082 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:08.678096 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:08.678457 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:08.678848 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:08.678853 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:08.678855 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:08.678994 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:08.679446 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:08.679453 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:08.679696 (+ 243us) client_negotiation.cc:769] Sending connection context
0926 16:32:08.679881 (+ 185us) client_negotiation.cc:240] Negotiation successful
0926 16:32:08.680183 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":104,"threads_started":1}
I20260926 16:32:08.682170 12060 heartbeater.cc:347] Connected to a master server at 127.11.8.254:35643
I20260926 16:32:08.682430 12060 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:08.682893 12060 heartbeater.cc:511] Master 127.11.8.254:35643 requested a full tablet report, sending...
I20260926 16:32:08.684481 11869 ts_manager.cc:194] Registered new tserver with Master: 9eda3ec510694f8c966192e03505c1d0 (127.11.8.193:34459)
I20260926 16:32:08.686084 11869 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.193@KRBTEST.COM'} at 127.11.8.193:44717
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab.
Entry for principal kudu/127.11.8.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab.
I20260926 16:32:08.693452 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.194:0
--local_ip_for_outbound_sockets=127.11.8.194
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:35643
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:36989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:08.801018 12065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:08.801258 12065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:08.801302 12065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:08.805135 12065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:08.805181 12065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:08.805238 12065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.194
I20260926 16:32:08.810143 12065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:36989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:35643
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.194
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:08.811110 12065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:08.812023 12065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:08.821206 12070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:08.821214 12071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:08.821281 12065 server_base.cc:1034] running on GCE node
W20260926 16:32:08.821214 12073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:08.821818 12065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:08.822518 12065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:08.823702 12065 hybrid_clock.cc:648] HybridClock initialized: now 1790440328823686 us; error 30 us; skew 500 ppm
Sep 26 16:32:08 dist-test-slave-2zvq krb5kdc[11824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440328, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:08.826438 12065 init.cc:377] Logged in from keytab as kudu/127.11.8.194@KRBTEST.COM (short username kudu)
I20260926 16:32:08.827270 12065 webserver.cc:460] Webserver started at http://127.11.8.194:34169/ using document root <none> and password file <none>
I20260926 16:32:08.827814 12065 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:08.827903 12065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:08.828105 12065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:08.830087 12065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-1/data/instance:
uuid: "e96b3cfb0f9c43139f98619519cb0ee0"
format_stamp: "Formatted at 2026-09-26 16:32:08 on dist-test-slave-2zvq"
server_key: "eb95c4922ab7da89e680a7a1150a8514"
server_key_iv: "97dac558321ed89c6e298ffd4c6f9275"
server_key_version: "encryptionkey@0"
I20260926 16:32:08.830713 12065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance:
uuid: "e96b3cfb0f9c43139f98619519cb0ee0"
format_stamp: "Formatted at 2026-09-26 16:32:08 on dist-test-slave-2zvq"
server_key: "eb95c4922ab7da89e680a7a1150a8514"
server_key_iv: "97dac558321ed89c6e298ffd4c6f9275"
server_key_version: "encryptionkey@0"
I20260926 16:32:08.834206 12065 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:32:08.836481 12080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:08.837651 12065 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260926 16:32:08.837782 12065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-1/wal
uuid: "e96b3cfb0f9c43139f98619519cb0ee0"
format_stamp: "Formatted at 2026-09-26 16:32:08 on dist-test-slave-2zvq"
server_key: "eb95c4922ab7da89e680a7a1150a8514"
server_key_iv: "97dac558321ed89c6e298ffd4c6f9275"
server_key_version: "encryptionkey@0"
I20260926 16:32:08.837888 12065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:08.849939 12065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:08.853011 12065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:08.853195 12065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:08.853713 12065 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:32:08.854512 12065 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:08.854580 12065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:08.854671 12065 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:08.854722 12065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:08.863262 12065 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.194:39297
I20260926 16:32:08.863291 12193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.194:39297 every 8 connection(s)
I20260926 16:32:08.864079 12065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
I20260926 16:32:08.869055 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 12065
I20260926 16:32:08.869170 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance
I20260926 16:32:08.869447 11299 external_mini_cluster.cc:1428] Setting key c1bfeeb8009df0a3ccaa8d8b3f20af3e
Sep 26 16:32:08 dist-test-slave-2zvq krb5kdc[11824](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440328, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:32:08.878830 11908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:08.865734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:41513 (local address 127.11.8.254:35643)
0926 16:32:08.865870 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:08.865873 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:08.866730 (+ 857us) server_negotiation.cc:407] Connection header received
0926 16:32:08.867988 (+ 1258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:08.867991 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:08.868049 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:08.868155 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:08.869637 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:08.870314 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:08.872010 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:08.872217 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:08.872296 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:08.875140 (+ 2844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:08.875143 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:08.875145 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:08.875169 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:08.876610 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:08.877247 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:08.877250 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:08.877251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:08.877323 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:08.877704 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:08.877707 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:08.877708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:08.877868 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:08.877946 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:08.878607 (+ 661us) server_negotiation.cc:299] Negotiation successful
0926 16:32:08.878702 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
WARNING: no policy specified for kudu/127.11.8.195@KRBTEST.COM; defaulting to no policy
I20260926 16:32:08.879987 12196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:08.866107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:35643 (local address 127.11.8.194:41513)
0926 16:32:08.866585 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:08.866627 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:08.867777 (+ 1150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:08.868316 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:08.868323 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:08.868813 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:08.869481 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:08.869503 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:08.870483 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:08.870486 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:08.871882 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:08.871897 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:08.872362 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:08.872368 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:08.872592 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:08.873229 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:08.873245 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:08.874974 (+ 1729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:08.876731 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:08.876740 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:08.876758 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:08.877137 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:08.877415 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:08.877419 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:08.877421 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:08.877608 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:08.877974 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:08.877980 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:08.878322 (+ 342us) client_negotiation.cc:769] Sending connection context
0926 16:32:08.878552 (+ 230us) client_negotiation.cc:240] Negotiation successful
0926 16:32:08.878901 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":164,"threads_started":1}
I20260926 16:32:08.881162 12194 heartbeater.cc:347] Connected to a master server at 127.11.8.254:35643
I20260926 16:32:08.881398 12194 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:08.881862 12194 heartbeater.cc:511] Master 127.11.8.254:35643 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.195@KRBTEST.COM" created.
I20260926 16:32:08.882949 11869 ts_manager.cc:194] Registered new tserver with Master: e96b3cfb0f9c43139f98619519cb0ee0 (127.11.8.194:39297)
I20260926 16:32:08.883524 11869 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.194@KRBTEST.COM'} at 127.11.8.194:41513
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab.
Entry for principal kudu/127.11.8.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab.
I20260926 16:32:08.896729 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.195:0
--local_ip_for_outbound_sockets=127.11.8.195
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:35643
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:36989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:09.006179 12199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:09.006474 12199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:09.006551 12199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:09.010633 12199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:09.010704 12199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:09.010809 12199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.195
I20260926 16:32:09.015944 12199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:36989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:35643
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.195
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:09.016939 12199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:09.017910 12199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:09.026988 12204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:09.027029 12205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:09.027010 12207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:09.027420 12199 server_base.cc:1034] running on GCE node
I20260926 16:32:09.027727 12199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:09.028252 12199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:09.029445 12199 hybrid_clock.cc:648] HybridClock initialized: now 1790440329029437 us; error 29 us; skew 500 ppm
Sep 26 16:32:09 dist-test-slave-2zvq krb5kdc[11824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440329, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:09.032196 12199 init.cc:377] Logged in from keytab as kudu/127.11.8.195@KRBTEST.COM (short username kudu)
I20260926 16:32:09.032977 12199 webserver.cc:460] Webserver started at http://127.11.8.195:34237/ using document root <none> and password file <none>
I20260926 16:32:09.033507 12199 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:09.033596 12199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:09.033797 12199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:09.035867 12199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-2/data/instance:
uuid: "878d605b053b4f208bf6534a3072122a"
format_stamp: "Formatted at 2026-09-26 16:32:09 on dist-test-slave-2zvq"
server_key: "a73c55041e913b91a9ce9ae1c26f54bf"
server_key_iv: "43211ac7a91047482dbb3d0780027e6b"
server_key_version: "encryptionkey@0"
I20260926 16:32:09.036473 12199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance:
uuid: "878d605b053b4f208bf6534a3072122a"
format_stamp: "Formatted at 2026-09-26 16:32:09 on dist-test-slave-2zvq"
server_key: "a73c55041e913b91a9ce9ae1c26f54bf"
server_key_iv: "43211ac7a91047482dbb3d0780027e6b"
server_key_version: "encryptionkey@0"
I20260926 16:32:09.040367 12199 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 16:32:09.042856 12214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:09.044039 12199 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:32:09.044162 12199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-2/wal
uuid: "878d605b053b4f208bf6534a3072122a"
format_stamp: "Formatted at 2026-09-26 16:32:09 on dist-test-slave-2zvq"
server_key: "a73c55041e913b91a9ce9ae1c26f54bf"
server_key_iv: "43211ac7a91047482dbb3d0780027e6b"
server_key_version: "encryptionkey@0"
I20260926 16:32:09.044260 12199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:09.061987 12199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:09.065146 12199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:09.065346 12199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:09.065896 12199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:32:09.066706 12199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:09.066779 12199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:09.066843 12199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:09.066886 12199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:09.075321 12199 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.195:43255
I20260926 16:32:09.075347 12327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.195:43255 every 8 connection(s)
I20260926 16:32:09.076195 12199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
I20260926 16:32:09.082855 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 12199
I20260926 16:32:09.082985 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance
I20260926 16:32:09.083262 11299 external_mini_cluster.cc:1428] Setting key 8d167f2e34bb11bb83e4b0cbe8457e95
Sep 26 16:32:09 dist-test-slave-2zvq krb5kdc[11824](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440329, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:32:09.089102 11908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:09.077880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:43359 (local address 127.11.8.254:35643)
0926 16:32:09.078017 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:09.078022 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:09.078930 (+ 908us) server_negotiation.cc:407] Connection header received
0926 16:32:09.080098 (+ 1168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:09.080102 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:09.080146 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:09.080220 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:09.081276 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:09.081819 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:09.083007 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:09.083331 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:09.083441 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:09.086131 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:09.086134 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:09.086136 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:09.086157 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:09.087427 (+ 1270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:09.087850 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:09.087853 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:09.087854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:09.087952 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:09.088230 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:09.088233 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:09.088234 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:09.088387 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:09.088462 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:09.088870 (+ 408us) server_negotiation.cc:299] Negotiation successful
0926 16:32:09.088991 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 16:32:09.089807 12330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:09.078360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:35643 (local address 127.11.8.195:43359)
0926 16:32:09.078785 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:09.078827 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:09.079908 (+ 1081us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:09.080372 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:09.080379 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:09.080713 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:09.081153 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:09.081166 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:09.081922 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:09.081925 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:09.082847 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:09.082856 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:09.083479 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:09.083482 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:09.083621 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:09.084264 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:09.084279 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:09.085988 (+ 1709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:09.087524 (+ 1536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:09.087528 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:09.087539 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:09.087767 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:09.088030 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:09.088032 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:09.088033 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:09.088154 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:09.088471 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:09.088476 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:09.088673 (+ 197us) client_negotiation.cc:769] Sending connection context
0926 16:32:09.088819 (+ 146us) client_negotiation.cc:240] Negotiation successful
0926 16:32:09.089046 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":97,"threads_started":1}
I20260926 16:32:09.090898 12328 heartbeater.cc:347] Connected to a master server at 127.11.8.254:35643
I20260926 16:32:09.091125 12328 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:09.091565 12328 heartbeater.cc:511] Master 127.11.8.254:35643 requested a full tablet report, sending...
I20260926 16:32:09.092585 11869 ts_manager.cc:194] Registered new tserver with Master: 878d605b053b4f208bf6534a3072122a (127.11.8.195:43255)
I20260926 16:32:09.093093 11869 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.195@KRBTEST.COM'} at 127.11.8.195:43359
I20260926 16:32:09.097136 11299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 16:32:09.107026 11908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:09.100488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38124 (local address 127.11.8.254:35643)
0926 16:32:09.100605 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:09.100608 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:09.100716 (+ 108us) server_negotiation.cc:407] Connection header received
0926 16:32:09.100801 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:09.100803 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:09.100837 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:09.100906 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:09.101566 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:09.102070 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:09.102941 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:09.103142 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:09.103218 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:09.104499 (+ 1281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:09.104501 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:09.104503 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:09.104521 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:09.105696 (+ 1175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:09.106060 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:09.106063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:09.106064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:09.106130 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:09.106360 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:09.106362 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:09.106363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:09.106543 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:09.106644 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:09.106830 (+ 186us) server_negotiation.cc:299] Negotiation successful
0926 16:32:09.106919 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 16:32:09.111421 11869 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38124:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:32:09.114210 11869 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:32:09.128108 12344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:09.123376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39046 (local address 127.11.8.195:43255)
0926 16:32:09.123800 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:09.123810 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:09.123903 (+ 93us) server_negotiation.cc:407] Connection header received
0926 16:32:09.124009 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:09.124014 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:09.124192 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:09.124347 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:09.124583 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:09.125480 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:09.126898 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:09.127430 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:09.127554 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:09.127691 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:09.127853 (+ 162us) server_negotiation.cc:299] Negotiation successful
0926 16:32:09.127954 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":145,"threads_started":1}
I20260926 16:32:09.128104 12340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:09.122873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:43255 (local address 127.0.0.1:39046)
0926 16:32:09.123800 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:09.123815 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:09.123908 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:09.124335 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:09.124338 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:09.124354 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:09.124457 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:09.124473 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:09.125609 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:09.125612 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:09.126794 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:09.126800 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:09.127557 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:09.127561 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:09.127661 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:09.127695 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 16:32:09.127826 (+ 131us) client_negotiation.cc:240] Negotiation successful
0926 16:32:09.127954 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":865,"mutex_wait_us":28,"thread_start_us":107,"threads_started":1}
I20260926 16:32:09.128261 12343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:09.122028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:39297 (local address 127.0.0.1:47926)
0926 16:32:09.123093 (+ 1065us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:09.123104 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:09.123212 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:09.123961 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:09.123966 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:09.123985 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:09.124076 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:09.124082 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:09.125397 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:09.125403 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:09.126631 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:09.126642 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:09.127758 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:09.127762 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:09.127904 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:09.127958 (+ 54us) client_negotiation.cc:769] Sending connection context
0926 16:32:09.128029 (+ 71us) client_negotiation.cc:240] Negotiation successful
0926 16:32:09.128116 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1015,"mutex_wait_us":40,"thread_start_us":85,"threads_started":1}
I20260926 16:32:09.128347 12341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:09.122222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36378 (local address 127.11.8.193:34459)
0926 16:32:09.122664 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:09.122680 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:09.122703 (+ 23us) server_negotiation.cc:407] Connection header received
0926 16:32:09.123033 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:09.123039 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:09.123244 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:09.123402 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:09.123897 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:09.125050 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:09.126605 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:09.127434 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:09.127554 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:09.127694 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:09.128103 (+ 409us) server_negotiation.cc:299] Negotiation successful
0926 16:32:09.128197 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":131,"threads_started":1}
I20260926 16:32:09.128714 12342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:09.122612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47926 (local address 127.11.8.194:39297)
0926 16:32:09.122906 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:09.122915 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:09.123193 (+ 278us) server_negotiation.cc:407] Connection header received
0926 16:32:09.123389 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:09.123395 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:09.123588 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:09.123710 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:09.124205 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:09.125276 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:09.126751 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:09.127495 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:09.127581 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:09.127702 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:09.128500 (+ 798us) server_negotiation.cc:299] Negotiation successful
0926 16:32:09.128594 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":119,"threads_started":1}
I20260926 16:32:09.128973 12339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:09.121943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:34459 (local address 127.0.0.1:36378)
0926 16:32:09.122397 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:09.122454 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:09.122580 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:09.123601 (+ 1021us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:09.123607 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:09.123631 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:09.123758 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:09.123765 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:09.125179 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:09.125182 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:09.126504 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:09.126517 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:09.127758 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:09.127762 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:09.127901 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:09.127959 (+ 58us) client_negotiation.cc:769] Sending connection context
0926 16:32:09.128029 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 16:32:09.128852 (+ 823us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":789,"thread_start_us":75,"threads_started":1}
I20260926 16:32:09.131503 11994 tablet_service.cc:1467] Processing CreateTablet for tablet 62365ea076c3491a8183b63cc9c2cb29 (DEFAULT_TABLE table=test-table [id=fda54bba615747f3982fe054adaae748]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:09.131510 12128 tablet_service.cc:1467] Processing CreateTablet for tablet 62365ea076c3491a8183b63cc9c2cb29 (DEFAULT_TABLE table=test-table [id=fda54bba615747f3982fe054adaae748]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:09.131510 12262 tablet_service.cc:1467] Processing CreateTablet for tablet 62365ea076c3491a8183b63cc9c2cb29 (DEFAULT_TABLE table=test-table [id=fda54bba615747f3982fe054adaae748]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:09.132498 11994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62365ea076c3491a8183b63cc9c2cb29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:09.132499 12262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62365ea076c3491a8183b63cc9c2cb29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:09.132499 12128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62365ea076c3491a8183b63cc9c2cb29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:09.138844 12345 tablet_bootstrap.cc:492] T 62365ea076c3491a8183b63cc9c2cb29 P 9eda3ec510694f8c966192e03505c1d0: Bootstrap starting.
I20260926 16:32:09.139998 12347 tablet_bootstrap.cc:492] T 62365ea076c3491a8183b63cc9c2cb29 P 878d605b053b4f208bf6534a3072122a: Bootstrap starting.
I20260926 16:32:09.140266 12346 tablet_bootstrap.cc:492] T 62365ea076c3491a8183b63cc9c2cb29 P e96b3cfb0f9c43139f98619519cb0ee0: Bootstrap starting.
I20260926 16:32:09.141214 12345 tablet_bootstrap.cc:654] T 62365ea076c3491a8183b63cc9c2cb29 P 9eda3ec510694f8c966192e03505c1d0: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:09.142158 12347 tablet_bootstrap.cc:654] T 62365ea076c3491a8183b63cc9c2cb29 P 878d605b053b4f208bf6534a3072122a: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:09.142238 12346 tablet_bootstrap.cc:654] T 62365ea076c3491a8183b63cc9c2cb29 P e96b3cfb0f9c43139f98619519cb0ee0: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:09.142428 12345 log.cc:824] T 62365ea076c3491a8183b63cc9c2cb29 P 9eda3ec510694f8c966192e03505c1d0: Log is configured to *not* fsync() on all Append() calls
I20260926 16:32:09.143422 12346 log.cc:824] T 62365ea076c3491a8183b63cc9c2cb29 P e96b3cfb0f9c43139f98619519cb0ee0: Log is configured to *not* fsync() on all Append() calls
I20260926 16:32:09.143419 12347 log.cc:824] T 62365ea076c3491a8183b63cc9c2cb29 P 878d605b053b4f208bf6534a3072122a: Log is configured to *not* fsync() on all Append() calls
I20260926 16:32:09.144526 12345 tablet_bootstrap.cc:492] T 62365ea076c3491a8183b63cc9c2cb29 P 9eda3ec510694f8c966192e03505c1d0: No bootstrap required, opened a new log
I20260926 16:32:09.144723 12345 ts_tablet_manager.cc:1397] T 62365ea076c3491a8183b63cc9c2cb29 P 9eda3ec510694f8c966192e03505c1d0: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260926 16:32:09.145457 12346 tablet_bootstrap.cc:492] T 62365ea076c3491a8183b63cc9c2cb29 P e96b3cfb0f9c43139f98619519cb0ee0: No bootstrap required, opened a new log
I20260926 16:32:09.145464 12347 tablet_bootstrap.cc:492] T 62365ea076c3491a8183b63cc9c2cb29 P 878d605b053b4f208bf6534a3072122a: No bootstrap required, opened a new log
I20260926 16:32:09.145629 12346 ts_tablet_manager.cc:1397] T 62365ea076c3491a8183b63cc9c2cb29 P e96b3cfb0f9c43139f98619519cb0ee0: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260926 16:32:09.145629 12347 ts_tablet_manager.cc:1397] T 62365ea076c3491a8183b63cc9c2cb29 P 878d605b053b4f208bf6534a3072122a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 16:32:09.147693 12345 raft_consensus.cc:348] T 62365ea076c3491a8183b63cc9c2cb29 P 9eda3ec510694f8c966192e03505c1d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eda3ec510694f8c966192e03505c1d0" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 34459 } } peers { permanent_uuid: "e96b3cfb0f9c43139f98619519cb0ee0" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 39297 } } peers { permanent_uuid: "878d605b053b4f208bf6534a3072122a" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 43255 } }
I20260926 16:32:09.147930 12345 raft_consensus.cc:374] T 62365ea076c3491a8183b63cc9c2cb29 P 9eda3ec510694f8c966192e03505c1d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:09.148052 12345 raft_consensus.cc:729] T 62365ea076c3491a8183b63cc9c2cb29 P 9eda3ec510694f8c966192e03505c1d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9eda3ec510694f8c966192e03505c1d0, State: Initialized, Role: FOLLOWER
I20260926 16:32:09.148551 12345 consensus_queue.cc:260] T 62365ea076c3491a8183b63cc9c2cb29 P 9eda3ec510694f8c966192e03505c1d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eda3ec510694f8c966192e03505c1d0" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 34459 } } peers { permanent_uuid: "e96b3cfb0f9c43139f98619519cb0ee0" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 39297 } } peers { permanent_uuid: "878d605b053b4f208bf6534a3072122a" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 43255 } }
I20260926 16:32:09.149463 12347 raft_consensus.cc:348] T 62365ea076c3491a8183b63cc9c2cb29 P 878d605b053b4f208bf6534a3072122a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eda3ec510694f8c966192e03505c1d0" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 34459 } } peers { permanent_uuid: "e96b3cfb0f9c43139f98619519cb0ee0" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 39297 } } peers { permanent_uuid: "878d605b053b4f208bf6534a3072122a" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 43255 } }
I20260926 16:32:09.149463 12346 raft_consensus.cc:348] T 62365ea076c3491a8183b63cc9c2cb29 P e96b3cfb0f9c43139f98619519cb0ee0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eda3ec510694f8c966192e03505c1d0" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 34459 } } peers { permanent_uuid: "e96b3cfb0f9c43139f98619519cb0ee0" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 39297 } } peers { permanent_uuid: "878d605b053b4f208bf6534a3072122a" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 43255 } }
I20260926 16:32:09.149719 12346 raft_consensus.cc:374] T 62365ea076c3491a8183b63cc9c2cb29 P e96b3cfb0f9c43139f98619519cb0ee0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:09.149739 12345 ts_tablet_manager.cc:1428] T 62365ea076c3491a8183b63cc9c2cb29 P 9eda3ec510694f8c966192e03505c1d0: Time spent starting tablet: real 0.005s user 0.000s sys 0.006s
I20260926 16:32:09.149748 12347 raft_consensus.cc:374] T 62365ea076c3491a8183b63cc9c2cb29 P 878d605b053b4f208bf6534a3072122a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:09.149744 12060 heartbeater.cc:503] Master 127.11.8.254:35643 was elected leader, sending a full tablet report...
I20260926 16:32:09.149796 12346 raft_consensus.cc:729] T 62365ea076c3491a8183b63cc9c2cb29 P e96b3cfb0f9c43139f98619519cb0ee0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e96b3cfb0f9c43139f98619519cb0ee0, State: Initialized, Role: FOLLOWER
I20260926 16:32:09.149809 12347 raft_consensus.cc:729] T 62365ea076c3491a8183b63cc9c2cb29 P 878d605b053b4f208bf6534a3072122a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878d605b053b4f208bf6534a3072122a, State: Initialized, Role: FOLLOWER
I20260926 16:32:09.150233 12346 consensus_queue.cc:260] T 62365ea076c3491a8183b63cc9c2cb29 P e96b3cfb0f9c43139f98619519cb0ee0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eda3ec510694f8c966192e03505c1d0" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 34459 } } peers { permanent_uuid: "e96b3cfb0f9c43139f98619519cb0ee0" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 39297 } } peers { permanent_uuid: "878d605b053b4f208bf6534a3072122a" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 43255 } }
I20260926 16:32:09.150523 12347 consensus_queue.cc:260] T 62365ea076c3491a8183b63cc9c2cb29 P 878d605b053b4f208bf6534a3072122a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eda3ec510694f8c966192e03505c1d0" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 34459 } } peers { permanent_uuid: "e96b3cfb0f9c43139f98619519cb0ee0" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 39297 } } peers { permanent_uuid: "878d605b053b4f208bf6534a3072122a" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 43255 } }
I20260926 16:32:09.150949 12194 heartbeater.cc:503] Master 127.11.8.254:35643 was elected leader, sending a full tablet report...
I20260926 16:32:09.151283 12346 ts_tablet_manager.cc:1428] T 62365ea076c3491a8183b63cc9c2cb29 P e96b3cfb0f9c43139f98619519cb0ee0: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260926 16:32:09.151314 12347 ts_tablet_manager.cc:1428] T 62365ea076c3491a8183b63cc9c2cb29 P 878d605b053b4f208bf6534a3072122a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260926 16:32:09.151614 12328 heartbeater.cc:503] Master 127.11.8.254:35643 was elected leader, sending a full tablet report...
W20260926 16:32:09.166014 12061 tablet.cc:2369] T 62365ea076c3491a8183b63cc9c2cb29 P 9eda3ec510694f8c966192e03505c1d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 16:32:09.327067 12329 tablet.cc:2369] T 62365ea076c3491a8183b63cc9c2cb29 P 878d605b053b4f208bf6534a3072122a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 16:32:09.365130 12195 tablet.cc:2369] T 62365ea076c3491a8183b63cc9c2cb29 P e96b3cfb0f9c43139f98619519cb0ee0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:32:09.436086 12352 raft_consensus.cc:482] T 62365ea076c3491a8183b63cc9c2cb29 P e96b3cfb0f9c43139f98619519cb0ee0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:32:09.436275 12352 raft_consensus.cc:504] T 62365ea076c3491a8183b63cc9c2cb29 P e96b3cfb0f9c43139f98619519cb0ee0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eda3ec510694f8c966192e03505c1d0" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 34459 } } peers { permanent_uuid: "e96b3cfb0f9c43139f98619519cb0ee0" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 39297 } } peers { permanent_uuid: "878d605b053b4f208bf6534a3072122a" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 43255 } }
I20260926 16:32:09.437353 12352 leader_election.cc:288] T 62365ea076c3491a8183b63cc9c2cb29 P e96b3cfb0f9c43139f98619519cb0ee0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9eda3ec510694f8c966192e03505c1d0 (127.11.8.193:34459), 878d605b053b4f208bf6534a3072122a (127.11.8.195:43255)
I20260926 16:32:09.441347 12341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:09.437653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:51995 (local address 127.11.8.193:34459)
0926 16:32:09.437806 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:09.437811 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:09.438048 (+ 237us) server_negotiation.cc:407] Connection header received
0926 16:32:09.438246 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:09.438250 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:09.438331 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:09.438389 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:09.438707 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:09.439292 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:09.440389 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:09.440770 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:09.440902 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:09.440956 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:09.441144 (+ 188us) server_negotiation.cc:299] Negotiation successful
0926 16:32:09.441211 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 16:32:09.441392 12354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:09.437573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:34459 (local address 127.11.8.194:51995)
0926 16:32:09.437942 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:09.437959 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:09.438092 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:09.438417 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:09.438420 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:09.438440 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:09.438579 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:09.438586 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:09.439407 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:09.439413 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:09.440259 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:09.440269 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:09.440903 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:09.440906 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:09.441010 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:09.441035 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 16:32:09.441128 (+ 93us) client_negotiation.cc:240] Negotiation successful
0926 16:32:09.441198 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"mutex_wait_us":23,"thread_start_us":88,"threads_started":1}
I20260926 16:32:09.441386 12344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:09.437712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:38483 (local address 127.11.8.195:43255)
0926 16:32:09.437913 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:09.437918 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:09.438147 (+ 229us) server_negotiation.cc:407] Connection header received
0926 16:32:09.438336 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:09.438339 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:09.438404 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:09.438488 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:09.438812 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:09.439393 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:09.440432 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:09.440832 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:09.440959 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:09.441011 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:09.441190 (+ 179us) server_negotiation.cc:299] Negotiation successful
0926 16:32:09.441266 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
I20260926 16:32:09.441435 12355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:09.437580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:43255 (local address 127.11.8.194:38483)
0926 16:32:09.438053 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:09.438065 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:09.438181 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:09.438511 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:09.438513 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:09.438527 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:09.438688 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:09.438694 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:09.439499 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:09.439505 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:09.440312 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:09.440321 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:09.440962 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:09.440965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:09.441074 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:09.441091 (+ 17us) client_negotiation.cc:769] Sending connection context
0926 16:32:09.441175 (+ 84us) client_negotiation.cc:240] Negotiation successful
0926 16:32:09.441241 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"mutex_wait_us":4,"thread_start_us":97,"threads_started":1}
I20260926 16:32:09.442060 12282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62365ea076c3491a8183b63cc9c2cb29" candidate_uuid: "e96b3cfb0f9c43139f98619519cb0ee0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878d605b053b4f208bf6534a3072122a" is_pre_election: true
I20260926 16:32:09.442103 12014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62365ea076c3491a8183b63cc9c2cb29" candidate_uuid: "e96b3cfb0f9c43139f98619519cb0ee0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9eda3ec510694f8c966192e03505c1d0" is_pre_election: true
I20260926 16:32:09.442406 12014 raft_consensus.cc:2454] T 62365ea076c3491a8183b63cc9c2cb29 P 9eda3ec510694f8c966192e03505c1d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e96b3cfb0f9c43139f98619519cb0ee0 in term 0.
I20260926 16:32:09.442404 12282 raft_consensus.cc:2454] T 62365ea076c3491a8183b63cc9c2cb29 P 878d605b053b4f208bf6534a3072122a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e96b3cfb0f9c43139f98619519cb0ee0 in term 0.
I20260926 16:32:09.442878 12081 leader_election.cc:302] T 62365ea076c3491a8183b63cc9c2cb29 P e96b3cfb0f9c43139f98619519cb0ee0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 878d605b053b4f208bf6534a3072122a, e96b3cfb0f9c43139f98619519cb0ee0; no voters:
I20260926 16:32:09.443164 12352 raft_consensus.cc:2781] T 62365ea076c3491a8183b63cc9c2cb29 P e96b3cfb0f9c43139f98619519cb0ee0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:32:09.443248 12352 raft_consensus.cc:482] T 62365ea076c3491a8183b63cc9c2cb29 P e96b3cfb0f9c43139f98619519cb0ee0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:32:09.443290 12352 raft_consensus.cc:3037] T 62365ea076c3491a8183b63cc9c2cb29 P e96b3cfb0f9c43139f98619519cb0ee0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:09.444370 12352 raft_consensus.cc:504] T 62365ea076c3491a8183b63cc9c2cb29 P e96b3cfb0f9c43139f98619519cb0ee0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eda3ec510694f8c966192e03505c1d0" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 34459 } } peers { permanent_uuid: "e96b3cfb0f9c43139f98619519cb0ee0" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 39297 } } peers { permanent_uuid: "878d605b053b4f208bf6534a3072122a" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 43255 } }
I20260926 16:32:09.444685 12352 leader_election.cc:288] T 62365ea076c3491a8183b63cc9c2cb29 P e96b3cfb0f9c43139f98619519cb0ee0 [CANDIDATE]: Term 1 election: Requested vote from peers 9eda3ec510694f8c966192e03505c1d0 (127.11.8.193:34459), 878d605b053b4f208bf6534a3072122a (127.11.8.195:43255)
I20260926 16:32:09.444986 12014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62365ea076c3491a8183b63cc9c2cb29" candidate_uuid: "e96b3cfb0f9c43139f98619519cb0ee0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9eda3ec510694f8c966192e03505c1d0"
I20260926 16:32:09.445027 12282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62365ea076c3491a8183b63cc9c2cb29" candidate_uuid: "e96b3cfb0f9c43139f98619519cb0ee0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878d605b053b4f208bf6534a3072122a"
I20260926 16:32:09.445111 12014 raft_consensus.cc:3037] T 62365ea076c3491a8183b63cc9c2cb29 P 9eda3ec510694f8c966192e03505c1d0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:09.445142 12282 raft_consensus.cc:3037] T 62365ea076c3491a8183b63cc9c2cb29 P 878d605b053b4f208bf6534a3072122a [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:09.446098 12014 raft_consensus.cc:2454] T 62365ea076c3491a8183b63cc9c2cb29 P 9eda3ec510694f8c966192e03505c1d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e96b3cfb0f9c43139f98619519cb0ee0 in term 1.
I20260926 16:32:09.446184 12282 raft_consensus.cc:2454] T 62365ea076c3491a8183b63cc9c2cb29 P 878d605b053b4f208bf6534a3072122a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e96b3cfb0f9c43139f98619519cb0ee0 in term 1.
I20260926 16:32:09.446404 12084 leader_election.cc:302] T 62365ea076c3491a8183b63cc9c2cb29 P e96b3cfb0f9c43139f98619519cb0ee0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9eda3ec510694f8c966192e03505c1d0, e96b3cfb0f9c43139f98619519cb0ee0; no voters:
I20260926 16:32:09.446583 12352 raft_consensus.cc:2781] T 62365ea076c3491a8183b63cc9c2cb29 P e96b3cfb0f9c43139f98619519cb0ee0 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:32:09.446832 12352 raft_consensus.cc:686] T 62365ea076c3491a8183b63cc9c2cb29 P e96b3cfb0f9c43139f98619519cb0ee0 [term 1 LEADER]: Becoming Leader. State: Replica: e96b3cfb0f9c43139f98619519cb0ee0, State: Running, Role: LEADER
I20260926 16:32:09.447146 12352 consensus_queue.cc:237] T 62365ea076c3491a8183b63cc9c2cb29 P e96b3cfb0f9c43139f98619519cb0ee0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eda3ec510694f8c966192e03505c1d0" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 34459 } } peers { permanent_uuid: "e96b3cfb0f9c43139f98619519cb0ee0" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 39297 } } peers { permanent_uuid: "878d605b053b4f208bf6534a3072122a" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 43255 } }
I20260926 16:32:09.450385 11869 catalog_manager.cc:5495] T 62365ea076c3491a8183b63cc9c2cb29 P e96b3cfb0f9c43139f98619519cb0ee0 reported cstate change: term changed from 0 to 1, leader changed from <none> to e96b3cfb0f9c43139f98619519cb0ee0 (127.11.8.194). New cstate: current_term: 1 leader_uuid: "e96b3cfb0f9c43139f98619519cb0ee0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9eda3ec510694f8c966192e03505c1d0" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 34459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e96b3cfb0f9c43139f98619519cb0ee0" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 39297 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "878d605b053b4f208bf6534a3072122a" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 43255 } health_report { overall_health: UNKNOWN } } }
Sep 26 16:32:09 dist-test-slave-2zvq krb5kdc[11824](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.193@KRBTEST.COM
I20260926 16:32:09.492739 12341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:09.483831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36380 (local address 127.11.8.193:34459)
0926 16:32:09.483999 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:09.484006 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:09.484193 (+ 187us) server_negotiation.cc:407] Connection header received
0926 16:32:09.484350 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:09.484354 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:09.484405 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:09.484502 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:09.485199 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:09.485713 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:09.486598 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:09.486818 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:09.486925 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:09.489354 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:09.489366 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:09.489381 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:09.489405 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:09.491004 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:09.491439 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:09.491445 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:09.491446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:09.491533 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:09.491807 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:09.491811 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:09.491812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:09.492058 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:09.492292 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:09.492464 (+ 172us) server_negotiation.cc:299] Negotiation successful
0926 16:32:09.492589 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Sep 26 16:32:09 dist-test-slave-2zvq krb5kdc[11824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440329, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 16:32:09 dist-test-slave-2zvq krb5kdc[11824](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440329, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.8.193@KRBTEST.COM
I20260926 16:32:09.518548 12341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:09.510852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36396 (local address 127.11.8.193:34459)
0926 16:32:09.511020 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:09.511024 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:09.511070 (+ 46us) server_negotiation.cc:407] Connection header received
0926 16:32:09.511180 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:09.511182 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:09.511224 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:09.511292 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:09.511937 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:09.512438 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:09.513221 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:09.513435 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:09.513508 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:09.515838 (+ 2330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:09.515841 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:09.515843 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:09.515864 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:09.517064 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:09.517428 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:09.517431 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:09.517432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:09.517500 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:09.517717 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:09.517720 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:09.517721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:09.517867 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:09.517962 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:09.518241 (+ 279us) server_negotiation.cc:299] Negotiation successful
0926 16:32:09.518427 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
Sep 26 16:32:09 dist-test-slave-2zvq krb5kdc[11824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 16:32:09 dist-test-slave-2zvq krb5kdc[11824](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.193@KRBTEST.COM
I20260926 16:32:09.542757 12341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:09.534515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36402 (local address 127.11.8.193:34459)
0926 16:32:09.534654 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:09.534662 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:09.534788 (+ 126us) server_negotiation.cc:407] Connection header received
0926 16:32:09.534898 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:09.534900 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:09.534939 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:09.535007 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:09.535612 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:09.536090 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:09.536879 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:09.537087 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:09.537164 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:09.539585 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:09.539588 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:09.539590 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:09.539611 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:09.541091 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:09.541472 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:09.541475 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:09.541476 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:09.541547 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:09.541806 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:09.541809 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:09.541810 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:09.541965 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:09.542098 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:09.542449 (+ 351us) server_negotiation.cc:299] Negotiation successful
0926 16:32:09.542571 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260926 16:32:09.544142 11974 tablet_service.cc:2908] Rejecting scan request for tablet 62365ea076c3491a8183b63cc9c2cb29: Uninitialized: clean time has not yet been initialized
I20260926 16:32:09.546255 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 11931
I20260926 16:32:09.552209 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 12065
I20260926 16:32:09.558184 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 12199
I20260926 16:32:09.563555 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 11837
2026-09-26T16:32:09Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2529 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:32:09 dist-test-slave-2zvq krb5kdc[12370](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:32:09 dist-test-slave-2zvq krb5kdc[12370](info): set up 2 sockets
Sep 26 16:32:09 dist-test-slave-2zvq krb5kdc[12370](info): commencing operation
krb5kdc: starting...
W20260926 16:32:10.591590 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:32:10 dist-test-slave-2zvq krb5kdc[12370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:32:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:32:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
I20260926 16:32:10.709262 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:36907
--webserver_interface=127.11.8.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:41379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:36907
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:10.817946 12383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:10.818261 12383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:10.818349 12383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:10.822304 12383 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:32:10.822407 12383 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 16:32:10.822439 12383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:10.822481 12383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:10.822517 12383 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:32:10.822554 12383 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:32:10.827669 12383 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:41379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:36907
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:36907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:10.828684 12383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:10.829684 12383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:10.837827 12388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:10.837803 12391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:10.837827 12389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:10.838053 12383 server_base.cc:1034] running on GCE node
I20260926 16:32:10.838730 12383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:10.839728 12383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:10.840917 12383 hybrid_clock.cc:648] HybridClock initialized: now 1790440330840902 us; error 29 us; skew 500 ppm
Sep 26 16:32:10 dist-test-slave-2zvq krb5kdc[12370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440330, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:10.848726 12383 init.cc:377] Logged in from keytab as kudu/127.11.8.254@KRBTEST.COM (short username kudu)
I20260926 16:32:10.849721 12383 webserver.cc:460] Webserver started at http://127.11.8.254:35863/ using document root <none> and password file <none>
I20260926 16:32:10.850409 12383 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:10.850514 12383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:10.850762 12383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:10.852876 12383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "5303c2fae4bb4b36befb691bd3c73723"
format_stamp: "Formatted at 2026-09-26 16:32:10 on dist-test-slave-2zvq"
server_key: "dbc8b0f9658fbc0e6b2474723d013441"
server_key_iv: "d8c53d505acfb2c6864ad107d104ef03"
server_key_version: "encryptionkey@0"
I20260926 16:32:10.853463 12383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "5303c2fae4bb4b36befb691bd3c73723"
format_stamp: "Formatted at 2026-09-26 16:32:10 on dist-test-slave-2zvq"
server_key: "dbc8b0f9658fbc0e6b2474723d013441"
server_key_iv: "d8c53d505acfb2c6864ad107d104ef03"
server_key_version: "encryptionkey@0"
I20260926 16:32:10.857210 12383 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260926 16:32:10.859635 12398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:10.860875 12383 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 16:32:10.861019 12383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "5303c2fae4bb4b36befb691bd3c73723"
format_stamp: "Formatted at 2026-09-26 16:32:10 on dist-test-slave-2zvq"
server_key: "dbc8b0f9658fbc0e6b2474723d013441"
server_key_iv: "d8c53d505acfb2c6864ad107d104ef03"
server_key_version: "encryptionkey@0"
I20260926 16:32:10.861127 12383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:10.871912 12383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:10.874747 12383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:10.874935 12383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:10.882382 12383 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:36907
I20260926 16:32:10.882405 12460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:36907 every 8 connection(s)
I20260926 16:32:10.883255 12383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:32:10.884944 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 12383
I20260926 16:32:10.885061 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:32:10.885321 11299 external_mini_cluster.cc:1428] Setting key f1e29ad34fa59624410e5e58172b1e6b
I20260926 16:32:10.886149 12461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:10.892517 12461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5303c2fae4bb4b36befb691bd3c73723: Bootstrap starting.
Sep 26 16:32:10 dist-test-slave-2zvq krb5kdc[12370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:32:10.895500 12461 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5303c2fae4bb4b36befb691bd3c73723: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:10.896768 12461 log.cc:824] T 00000000000000000000000000000000 P 5303c2fae4bb4b36befb691bd3c73723: Log is configured to *not* fsync() on all Append() calls
I20260926 16:32:10.899662 12461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5303c2fae4bb4b36befb691bd3c73723: No bootstrap required, opened a new log
I20260926 16:32:10.900398 12464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:10.886707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47064 (local address 127.11.8.254:36907)
0926 16:32:10.887129 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:10.887143 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:10.887177 (+ 34us) server_negotiation.cc:407] Connection header received
0926 16:32:10.887854 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:10.887870 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:10.888190 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:10.888521 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:10.889311 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:10.890362 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:10.891232 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:10.891713 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:10.891883 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:10.894799 (+ 2916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:10.894804 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:10.894818 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:10.894845 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:10.896677 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:10.897185 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:10.897191 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:10.897195 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:10.897305 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:10.897642 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:10.897649 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:10.897655 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:10.898240 (+ 585us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:10.898474 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:10.898879 (+ 405us) server_negotiation.cc:299] Negotiation successful
0926 16:32:10.899354 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":148,"threads_started":1}
I20260926 16:32:10.903359 12461 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5303c2fae4bb4b36befb691bd3c73723 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5303c2fae4bb4b36befb691bd3c73723" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 36907 } }
I20260926 16:32:10.903581 12461 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5303c2fae4bb4b36befb691bd3c73723 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:10.903616 12461 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5303c2fae4bb4b36befb691bd3c73723 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5303c2fae4bb4b36befb691bd3c73723, State: Initialized, Role: FOLLOWER
I20260926 16:32:10.903977 12461 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5303c2fae4bb4b36befb691bd3c73723 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5303c2fae4bb4b36befb691bd3c73723" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 36907 } }
I20260926 16:32:10.904085 12461 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5303c2fae4bb4b36befb691bd3c73723 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:32:10.904122 12461 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5303c2fae4bb4b36befb691bd3c73723 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:32:10.904191 12461 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5303c2fae4bb4b36befb691bd3c73723 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:10.905076 12461 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5303c2fae4bb4b36befb691bd3c73723 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5303c2fae4bb4b36befb691bd3c73723" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 36907 } }
I20260926 16:32:10.905367 12461 leader_election.cc:302] T 00000000000000000000000000000000 P 5303c2fae4bb4b36befb691bd3c73723 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5303c2fae4bb4b36befb691bd3c73723; no voters:
I20260926 16:32:10.905642 12461 leader_election.cc:288] T 00000000000000000000000000000000 P 5303c2fae4bb4b36befb691bd3c73723 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:32:10.905886 12466 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5303c2fae4bb4b36befb691bd3c73723 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:32:10.906165 12466 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5303c2fae4bb4b36befb691bd3c73723 [term 1 LEADER]: Becoming Leader. State: Replica: 5303c2fae4bb4b36befb691bd3c73723, State: Running, Role: LEADER
I20260926 16:32:10.906555 12466 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5303c2fae4bb4b36befb691bd3c73723 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5303c2fae4bb4b36befb691bd3c73723" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 36907 } }
I20260926 16:32:10.906989 12461 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5303c2fae4bb4b36befb691bd3c73723 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:32:10.908247 12468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5303c2fae4bb4b36befb691bd3c73723 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5303c2fae4bb4b36befb691bd3c73723. Latest consensus state: current_term: 1 leader_uuid: "5303c2fae4bb4b36befb691bd3c73723" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5303c2fae4bb4b36befb691bd3c73723" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 36907 } } }
I20260926 16:32:10.908378 12468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5303c2fae4bb4b36befb691bd3c73723 [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:10.908747 12475 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:32:10.908717 12467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5303c2fae4bb4b36befb691bd3c73723 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5303c2fae4bb4b36befb691bd3c73723" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5303c2fae4bb4b36befb691bd3c73723" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 36907 } } }
I20260926 16:32:10.908859 12467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5303c2fae4bb4b36befb691bd3c73723 [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:10.911652 12475 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:32:10.916704 12475 catalog_manager.cc:1347] Generated new cluster ID: 80ad90428b86400fab3d75ab11f84878
I20260926 16:32:10.916771 12475 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:32:10.934336 12475 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:32:10.935714 12475 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:32:10.940393 12475 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5303c2fae4bb4b36befb691bd3c73723: Generated new TSK 0
I20260926 16:32:10.941205 12475 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.8.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab.
Entry for principal kudu/127.11.8.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab.
I20260926 16:32:10.966898 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.193:0
--local_ip_for_outbound_sockets=127.11.8.193
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:36907
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:41379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:11.075798 12487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:11.076059 12487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:11.076130 12487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:11.080024 12487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:11.080101 12487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:11.080168 12487 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 16:32:11.080245 12487 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.193
I20260926 16:32:11.085280 12487 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:41379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.8.254:36907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.193
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:11.086254 12487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:11.087244 12487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 16:32:11.097209 12487 server_base.cc:1034] running on GCE node
W20260926 16:32:11.097194 12495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:11.097218 12493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:11.097199 12492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:11.098125 12487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:11.098783 12487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:11.099946 12487 hybrid_clock.cc:648] HybridClock initialized: now 1790440331099924 us; error 33 us; skew 500 ppm
Sep 26 16:32:11 dist-test-slave-2zvq krb5kdc[12370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440331, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:11.102774 12487 init.cc:377] Logged in from keytab as kudu/127.11.8.193@KRBTEST.COM (short username kudu)
I20260926 16:32:11.103590 12487 webserver.cc:460] Webserver started at http://127.11.8.193:41289/ using document root <none> and password file <none>
I20260926 16:32:11.104128 12487 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:11.104187 12487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:11.104406 12487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:11.106410 12487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-0/data/instance:
uuid: "af9cfd84a2ad48f6864a099828976e6a"
format_stamp: "Formatted at 2026-09-26 16:32:11 on dist-test-slave-2zvq"
server_key: "afc1e4f0529cb4e983ad5b979eac63cf"
server_key_iv: "2bd487c97fbed7f4c285986a8e88deef"
server_key_version: "encryptionkey@0"
I20260926 16:32:11.106953 12487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance:
uuid: "af9cfd84a2ad48f6864a099828976e6a"
format_stamp: "Formatted at 2026-09-26 16:32:11 on dist-test-slave-2zvq"
server_key: "afc1e4f0529cb4e983ad5b979eac63cf"
server_key_iv: "2bd487c97fbed7f4c285986a8e88deef"
server_key_version: "encryptionkey@0"
I20260926 16:32:11.110689 12487 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:32:11.112977 12502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:11.114126 12487 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:32:11.114310 12487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-0/wal
uuid: "af9cfd84a2ad48f6864a099828976e6a"
format_stamp: "Formatted at 2026-09-26 16:32:11 on dist-test-slave-2zvq"
server_key: "afc1e4f0529cb4e983ad5b979eac63cf"
server_key_iv: "2bd487c97fbed7f4c285986a8e88deef"
server_key_version: "encryptionkey@0"
I20260926 16:32:11.114456 12487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:11.137476 12487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:11.140445 12487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:11.140640 12487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:11.142030 12487 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:11.142103 12487 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:11.142179 12487 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:11.142222 12487 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 16:32:11 dist-test-slave-2zvq krb5kdc[12370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440331, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:32:11.152864 12487 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.193:40449
I20260926 16:32:11.152957 12617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.193:40449 every 8 connection(s)
I20260926 16:32:11.153721 12487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
I20260926 16:32:11.155598 12464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:11.142842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:37395 (local address 127.11.8.254:36907)
0926 16:32:11.143048 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:11.143053 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:11.143853 (+ 800us) server_negotiation.cc:407] Connection header received
0926 16:32:11.144955 (+ 1102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:11.144961 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:11.145028 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:11.145089 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:11.146589 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.147346 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.148554 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.148806 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.148924 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:11.151934 (+ 3010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:11.151938 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:11.151941 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:11.151967 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:11.153582 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:11.154114 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:11.154119 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:11.154121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:11.154215 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:11.154658 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:11.154662 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:11.154664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:11.154874 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:11.154969 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:11.155372 (+ 403us) server_negotiation.cc:299] Negotiation successful
0926 16:32:11.155484 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 16:32:11.156322 12511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:11.143244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36907 (local address 127.11.8.193:37395)
0926 16:32:11.143686 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:11.143734 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:11.144682 (+ 948us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:11.145436 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:11.145442 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:11.145821 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:11.146447 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:11.146472 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.147483 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.147486 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:11.148438 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:11.148447 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.148975 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.148979 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:11.149138 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:11.149669 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:11.149684 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:11.151753 (+ 2069us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:11.153700 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:11.153705 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:11.153719 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:11.154015 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:11.154347 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:11.154350 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:11.154352 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:11.154521 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:11.154966 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:11.154970 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:11.155175 (+ 205us) client_negotiation.cc:769] Sending connection context
0926 16:32:11.155328 (+ 153us) client_negotiation.cc:240] Negotiation successful
0926 16:32:11.155563 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":138,"threads_started":1}
I20260926 16:32:11.157481 12618 heartbeater.cc:347] Connected to a master server at 127.11.8.254:36907
I20260926 16:32:11.157636 12618 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:11.158085 12618 heartbeater.cc:511] Master 127.11.8.254:36907 requested a full tablet report, sending...
I20260926 16:32:11.159647 12414 ts_manager.cc:194] Registered new tserver with Master: af9cfd84a2ad48f6864a099828976e6a (127.11.8.193:40449)
I20260926 16:32:11.160668 12414 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.193@KRBTEST.COM'} at 127.11.8.193:37395
I20260926 16:32:11.163390 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 12487
I20260926 16:32:11.163524 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance
I20260926 16:32:11.163818 11299 external_mini_cluster.cc:1428] Setting key 85ebceda78b69ec3a98771bdb48649e5
I20260926 16:32:11.167861 12624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:11.159415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36907 (local address 127.11.8.193:48437)
0926 16:32:11.161268 (+ 1853us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:11.161281 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:11.161383 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:11.161621 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:11.161624 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:11.161803 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:11.161904 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:11.161910 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.162698 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.162702 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:11.163267 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:11.163275 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.163802 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.163805 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:11.163908 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:11.164392 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:11.164413 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:11.164686 (+ 273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:11.166434 (+ 1748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:11.166440 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:11.166445 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:11.166710 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:11.167023 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:11.167025 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:11.167026 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:11.167096 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:11.167499 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:11.167501 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:11.167569 (+ 68us) client_negotiation.cc:769] Sending connection context
0926 16:32:11.167645 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 16:32:11.167741 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1770,"thread_start_us":95,"threads_started":1}
I20260926 16:32:11.168097 12464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:11.159484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:48437 (local address 127.11.8.254:36907)
0926 16:32:11.159594 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:11.159597 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:11.161345 (+ 1748us) server_negotiation.cc:407] Connection header received
0926 16:32:11.161481 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:11.161483 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:11.161521 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:11.161606 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:11.162010 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.162580 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.163377 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.163694 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.163797 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:11.164809 (+ 1012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:11.164813 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:11.164815 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:11.164838 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:11.166335 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:11.166817 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:11.166823 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:11.166826 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:11.166919 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:11.167181 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:11.167186 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:11.167190 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:11.167409 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:11.167514 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:11.167727 (+ 213us) server_negotiation.cc:299] Negotiation successful
0926 16:32:11.167859 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
WARNING: no policy specified for kudu/127.11.8.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab.
Entry for principal kudu/127.11.8.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab.
I20260926 16:32:11.189154 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.194:0
--local_ip_for_outbound_sockets=127.11.8.194
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:36907
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:41379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:11.303347 12627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:11.303718 12627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:11.303803 12627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:11.308271 12627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:11.308331 12627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:11.308400 12627 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 16:32:11.308473 12627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.194
I20260926 16:32:11.313812 12627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:41379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.8.254:36907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.194
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:11.314890 12627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:11.315886 12627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:11.325433 12635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:11.325484 12633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:11.325492 12632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:11.325687 12627 server_base.cc:1034] running on GCE node
I20260926 16:32:11.326180 12627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:11.326844 12627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:11.328027 12627 hybrid_clock.cc:648] HybridClock initialized: now 1790440331328026 us; error 41 us; skew 500 ppm
Sep 26 16:32:11 dist-test-slave-2zvq krb5kdc[12370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440331, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:11.330731 12627 init.cc:377] Logged in from keytab as kudu/127.11.8.194@KRBTEST.COM (short username kudu)
I20260926 16:32:11.331615 12627 webserver.cc:460] Webserver started at http://127.11.8.194:45821/ using document root <none> and password file <none>
I20260926 16:32:11.332182 12627 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:11.332244 12627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:11.332504 12627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:11.334715 12627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-1/data/instance:
uuid: "c84b627d26e04df5b0550e4f520f764a"
format_stamp: "Formatted at 2026-09-26 16:32:11 on dist-test-slave-2zvq"
server_key: "d8612bb1dd81a31e6a7297a0c1c71e40"
server_key_iv: "5f36ab6709e32f1d04484464d25e82ee"
server_key_version: "encryptionkey@0"
I20260926 16:32:11.335366 12627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance:
uuid: "c84b627d26e04df5b0550e4f520f764a"
format_stamp: "Formatted at 2026-09-26 16:32:11 on dist-test-slave-2zvq"
server_key: "d8612bb1dd81a31e6a7297a0c1c71e40"
server_key_iv: "5f36ab6709e32f1d04484464d25e82ee"
server_key_version: "encryptionkey@0"
I20260926 16:32:11.339198 12627 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 16:32:11.341602 12642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:11.342880 12627 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:32:11.343003 12627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-1/wal
uuid: "c84b627d26e04df5b0550e4f520f764a"
format_stamp: "Formatted at 2026-09-26 16:32:11 on dist-test-slave-2zvq"
server_key: "d8612bb1dd81a31e6a7297a0c1c71e40"
server_key_iv: "5f36ab6709e32f1d04484464d25e82ee"
server_key_version: "encryptionkey@0"
I20260926 16:32:11.343103 12627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:11.354240 12627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:11.356935 12627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:11.357116 12627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:11.358566 12627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:11.358614 12627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:11.358673 12627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:11.358704 12627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 16:32:11 dist-test-slave-2zvq krb5kdc[12370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440331, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:32:11.369725 12627 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.194:44717
I20260926 16:32:11.369784 12757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.194:44717 every 8 connection(s)
I20260926 16:32:11.370891 12627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
I20260926 16:32:11.372768 12464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:11.359468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:42963 (local address 127.11.8.254:36907)
0926 16:32:11.359601 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:11.359605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:11.360552 (+ 947us) server_negotiation.cc:407] Connection header received
0926 16:32:11.361505 (+ 953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:11.361509 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:11.361573 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:11.361633 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:11.363207 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.364046 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.365582 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.366010 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.366143 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:11.368823 (+ 2680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:11.368827 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:11.368830 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:11.368858 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:11.370597 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:11.371125 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:11.371130 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:11.371131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:11.371219 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:11.371609 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:11.371615 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:11.371618 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:11.371835 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:11.371954 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:11.372460 (+ 506us) server_negotiation.cc:299] Negotiation successful
0926 16:32:11.372594 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 16:32:11.373603 12651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:11.359945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36907 (local address 127.11.8.194:42963)
0926 16:32:11.360404 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:11.360447 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:11.361238 (+ 791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:11.362116 (+ 878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:11.362128 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:11.362601 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:11.363052 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:11.363066 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.364175 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.364178 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:11.365427 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:11.365438 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.366193 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.366199 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:11.366428 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:11.367051 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:11.367067 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:11.368655 (+ 1588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:11.370707 (+ 2052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:11.370711 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:11.370722 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:11.371010 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:11.371321 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:11.371324 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:11.371326 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:11.371499 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:11.371951 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:11.371957 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:11.372205 (+ 248us) client_negotiation.cc:769] Sending connection context
0926 16:32:11.372423 (+ 218us) client_negotiation.cc:240] Negotiation successful
0926 16:32:11.372739 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":135,"threads_started":1}
I20260926 16:32:11.374917 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 12627
I20260926 16:32:11.375036 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance
I20260926 16:32:11.375299 11299 external_mini_cluster.cc:1428] Setting key f24b019bf7ab89344058bd8aebed346a
I20260926 16:32:11.377036 12758 heartbeater.cc:347] Connected to a master server at 127.11.8.254:36907
I20260926 16:32:11.377177 12758 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:11.377676 12758 heartbeater.cc:511] Master 127.11.8.254:36907 requested a full tablet report, sending...
I20260926 16:32:11.378731 12415 ts_manager.cc:194] Registered new tserver with Master: c84b627d26e04df5b0550e4f520f764a (127.11.8.194:44717)
I20260926 16:32:11.379576 12415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.194@KRBTEST.COM'} at 127.11.8.194:42963
I20260926 16:32:11.384945 12464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:11.377242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:40337 (local address 127.11.8.254:36907)
0926 16:32:11.377371 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:11.377377 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:11.377762 (+ 385us) server_negotiation.cc:407] Connection header received
0926 16:32:11.377900 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:11.377903 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:11.377948 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:11.378029 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:11.378589 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.379118 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.379998 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.380258 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.380363 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:11.381768 (+ 1405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:11.381771 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:11.381772 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:11.381792 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:11.383313 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:11.383787 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:11.383789 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:11.383791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:11.383881 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:11.384170 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:11.384172 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:11.384173 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:11.384354 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:11.384430 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:11.384746 (+ 316us) server_negotiation.cc:299] Negotiation successful
0926 16:32:11.384834 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260926 16:32:11.385038 12764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:11.377356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36907 (local address 127.11.8.194:40337)
0926 16:32:11.377657 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:11.377673 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:11.377775 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:11.378061 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:11.378067 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:11.378327 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:11.378480 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:11.378490 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.379239 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.379243 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:11.379881 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:11.379900 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.380381 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.380395 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:11.380529 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:11.381113 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:11.381132 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:11.381627 (+ 495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:11.383432 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:11.383435 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:11.383437 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:11.383693 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:11.383968 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:11.383973 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:11.383977 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:11.384081 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:11.384452 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:11.384456 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:11.384552 (+ 96us) client_negotiation.cc:769] Sending connection context
0926 16:32:11.384673 (+ 121us) client_negotiation.cc:240] Negotiation successful
0926 16:32:11.384818 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":104,"threads_started":1}
WARNING: no policy specified for kudu/127.11.8.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab.
Entry for principal kudu/127.11.8.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab.
I20260926 16:32:11.401096 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.195:0
--local_ip_for_outbound_sockets=127.11.8.195
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:36907
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:41379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:11.510246 12767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:11.510507 12767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:11.510548 12767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:11.514431 12767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:11.514482 12767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:11.514518 12767 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 16:32:11.514549 12767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.195
I20260926 16:32:11.519771 12767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:41379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.8.254:36907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.195
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:11.520689 12767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:11.521631 12767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 16:32:11.531080 12767 server_base.cc:1034] running on GCE node
W20260926 16:32:11.530948 12772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:11.530968 12773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:11.530948 12775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:11.531598 12767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:11.532248 12767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:11.533440 12767 hybrid_clock.cc:648] HybridClock initialized: now 1790440331533421 us; error 35 us; skew 500 ppm
Sep 26 16:32:11 dist-test-slave-2zvq krb5kdc[12370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440331, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:11.536180 12767 init.cc:377] Logged in from keytab as kudu/127.11.8.195@KRBTEST.COM (short username kudu)
I20260926 16:32:11.537096 12767 webserver.cc:460] Webserver started at http://127.11.8.195:38477/ using document root <none> and password file <none>
I20260926 16:32:11.537662 12767 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:11.537752 12767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:11.538233 12767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:11.540336 12767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-2/data/instance:
uuid: "4ff1455b40274737b068e75621fc0fb9"
format_stamp: "Formatted at 2026-09-26 16:32:11 on dist-test-slave-2zvq"
server_key: "609b8f715001185fd7585396d17dde3c"
server_key_iv: "720f1a3954de828893b69f247695399f"
server_key_version: "encryptionkey@0"
I20260926 16:32:11.540937 12767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance:
uuid: "4ff1455b40274737b068e75621fc0fb9"
format_stamp: "Formatted at 2026-09-26 16:32:11 on dist-test-slave-2zvq"
server_key: "609b8f715001185fd7585396d17dde3c"
server_key_iv: "720f1a3954de828893b69f247695399f"
server_key_version: "encryptionkey@0"
I20260926 16:32:11.544584 12767 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 16:32:11.547037 12782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:11.548237 12767 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:32:11.548372 12767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-2/wal
uuid: "4ff1455b40274737b068e75621fc0fb9"
format_stamp: "Formatted at 2026-09-26 16:32:11 on dist-test-slave-2zvq"
server_key: "609b8f715001185fd7585396d17dde3c"
server_key_iv: "720f1a3954de828893b69f247695399f"
server_key_version: "encryptionkey@0"
I20260926 16:32:11.548477 12767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:11.562176 12767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:11.565550 12767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:11.565740 12767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:11.567272 12767 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:11.567342 12767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:11.567411 12767 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:11.567459 12767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 16:32:11 dist-test-slave-2zvq krb5kdc[12370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440331, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:32:11.578464 12767 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.195:33629
I20260926 16:32:11.578512 12897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.195:33629 every 8 connection(s)
I20260926 16:32:11.579396 12767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
I20260926 16:32:11.582134 12464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:11.568041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:58221 (local address 127.11.8.254:36907)
0926 16:32:11.568188 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:11.568193 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:11.569148 (+ 955us) server_negotiation.cc:407] Connection header received
0926 16:32:11.570458 (+ 1310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:11.570462 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:11.570529 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:11.570647 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:11.572042 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.572944 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.574411 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.574710 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.574835 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:11.578670 (+ 3835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:11.578674 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:11.578677 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:11.578714 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:11.580369 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:11.580848 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:11.580851 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:11.580852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:11.580928 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:11.581213 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:11.581216 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:11.581217 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:11.581386 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:11.581457 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:11.581923 (+ 466us) server_negotiation.cc:299] Negotiation successful
0926 16:32:11.582019 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 16:32:11.582998 12791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:11.568474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36907 (local address 127.11.8.195:58221)
0926 16:32:11.568964 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:11.569021 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:11.570157 (+ 1136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:11.570829 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:11.570836 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:11.571165 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:11.571838 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:11.571867 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.573092 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.573097 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:11.574123 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:11.574132 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.574887 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.574891 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:11.575141 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:11.576478 (+ 1337us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:11.576502 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:11.578451 (+ 1949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:11.580464 (+ 2013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:11.580468 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:11.580478 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:11.580720 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:11.581006 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:11.581009 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:11.581010 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:11.581133 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:11.581468 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:11.581473 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:11.581684 (+ 211us) client_negotiation.cc:769] Sending connection context
0926 16:32:11.581828 (+ 144us) client_negotiation.cc:240] Negotiation successful
0926 16:32:11.582098 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":109,"threads_started":1}
I20260926 16:32:11.584219 12898 heartbeater.cc:347] Connected to a master server at 127.11.8.254:36907
I20260926 16:32:11.584369 12898 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:11.584848 12898 heartbeater.cc:511] Master 127.11.8.254:36907 requested a full tablet report, sending...
I20260926 16:32:11.585987 12414 ts_manager.cc:194] Registered new tserver with Master: 4ff1455b40274737b068e75621fc0fb9 (127.11.8.195:33629)
I20260926 16:32:11.586802 12414 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.195@KRBTEST.COM'} at 127.11.8.195:58221
I20260926 16:32:11.587065 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 12767
I20260926 16:32:11.587190 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance
I20260926 16:32:11.587509 11299 external_mini_cluster.cc:1428] Setting key 4ab1a55b7a2b3275fd7279bcfb57f416
I20260926 16:32:11.589447 11299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 16:32:11.593955 12904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:11.586221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36907 (local address 127.11.8.195:54065)
0926 16:32:11.586535 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:11.586547 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:11.586623 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:11.586859 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:11.586905 (+ 46us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:11.587110 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:11.587305 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:11.587314 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.588337 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.588340 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:11.589060 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:11.589069 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.589519 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.589522 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:11.589611 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:11.590107 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:11.590120 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:11.590494 (+ 374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:11.592474 (+ 1980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:11.592477 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:11.592479 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:11.592664 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:11.592955 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:11.592957 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:11.592958 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:11.593030 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:11.593474 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:11.593476 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:11.593549 (+ 73us) client_negotiation.cc:769] Sending connection context
0926 16:32:11.593627 (+ 78us) client_negotiation.cc:240] Negotiation successful
0926 16:32:11.593727 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":124,"threads_started":1}
I20260926 16:32:11.594065 12464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:11.586351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:54065 (local address 127.11.8.254:36907)
0926 16:32:11.586476 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:11.586480 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:11.586610 (+ 130us) server_negotiation.cc:407] Connection header received
0926 16:32:11.586683 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:11.586686 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:11.586724 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:11.586803 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:11.587412 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.588134 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.589178 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.589415 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.589518 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:11.590630 (+ 1112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:11.590634 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:11.590636 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:11.590658 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:11.592358 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:11.592763 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:11.592769 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:11.592774 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:11.592864 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:11.593121 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:11.593127 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:11.593131 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:11.593377 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:11.593483 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:11.593716 (+ 233us) server_negotiation.cc:299] Negotiation successful
0926 16:32:11.593843 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
Sep 26 16:32:11 dist-test-slave-2zvq krb5kdc[12370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440331, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 16:32:11 dist-test-slave-2zvq krb5kdc[12370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440331, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:32:11.616364 12464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:11.608262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47068 (local address 127.11.8.254:36907)
0926 16:32:11.608395 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:11.608399 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:11.608411 (+ 12us) server_negotiation.cc:407] Connection header received
0926 16:32:11.608451 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:11.608453 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:11.608490 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:11.608554 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:11.609312 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.609919 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.610832 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.611067 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.611148 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:11.613600 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:11.613603 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:11.613605 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:11.613632 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:11.614910 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:11.615287 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:11.615289 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:11.615290 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:11.615359 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:11.615600 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:11.615602 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:11.615603 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:11.615764 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:11.615862 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:11.616056 (+ 194us) server_negotiation.cc:299] Negotiation successful
0926 16:32:11.616159 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 16:32:11.619424 12414 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47068:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:32:11.622423 12414 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:32:11.636734 12917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:11.630680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33372 (local address 127.11.8.195:33629)
0926 16:32:11.631197 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:11.631209 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:11.631555 (+ 346us) server_negotiation.cc:407] Connection header received
0926 16:32:11.631774 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:11.631779 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:11.631988 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:11.632117 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:11.632562 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.633700 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.635532 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.636073 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.636199 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:11.636360 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:11.636488 (+ 128us) server_negotiation.cc:299] Negotiation successful
0926 16:32:11.636583 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":106,"threads_started":1}
I20260926 16:32:11.636722 12919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:11.630475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:33629 (local address 127.0.0.1:33372)
0926 16:32:11.631421 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:11.631472 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:11.631609 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:11.632290 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:11.632293 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:11.632310 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:11.632447 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:11.632454 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.633838 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.633842 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:11.635411 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:11.635421 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.636196 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.636200 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:11.636311 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:11.636351 (+ 40us) client_negotiation.cc:769] Sending connection context
0926 16:32:11.636460 (+ 109us) client_negotiation.cc:240] Negotiation successful
0926 16:32:11.636583 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":821,"mutex_wait_us":701,"thread_start_us":92,"threads_started":1}
I20260926 16:32:11.636885 12916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:11.630656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42488 (local address 127.11.8.194:44717)
0926 16:32:11.631315 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:11.631326 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:11.631734 (+ 408us) server_negotiation.cc:407] Connection header received
0926 16:32:11.631799 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:11.631804 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:11.631988 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:11.632127 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:11.632488 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.633698 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.635415 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.636179 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.636268 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:11.636393 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:11.636660 (+ 267us) server_negotiation.cc:299] Negotiation successful
0926 16:32:11.636754 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":142,"threads_started":1}
I20260926 16:32:11.636874 12915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:11.630475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:44717 (local address 127.0.0.1:42488)
0926 16:32:11.631421 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:11.631472 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:11.631602 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:11.632156 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:11.632163 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:11.632186 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:11.632358 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:11.632367 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.633838 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.633842 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:11.635290 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:11.635301 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.636457 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.636460 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:11.636523 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:11.636540 (+ 17us) client_negotiation.cc:769] Sending connection context
0926 16:32:11.636629 (+ 89us) client_negotiation.cc:240] Negotiation successful
0926 16:32:11.636698 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":783,"mutex_wait_us":111,"thread_start_us":72,"threads_started":1}
I20260926 16:32:11.637403 12918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:11.631150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:40449 (local address 127.0.0.1:50666)
0926 16:32:11.632203 (+ 1053us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:11.632216 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:11.632308 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:11.632737 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:11.632740 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:11.632756 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:11.632848 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:11.632854 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.633769 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.633774 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:11.635088 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:11.635101 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.637028 (+ 1927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.637032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:11.637126 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:11.637147 (+ 21us) client_negotiation.cc:769] Sending connection context
0926 16:32:11.637215 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 16:32:11.637289 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":999,"mutex_wait_us":43,"thread_start_us":96,"threads_started":1}
I20260926 16:32:11.637797 12920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:11.631641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50666 (local address 127.11.8.193:40449)
0926 16:32:11.632053 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:11.632068 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:11.632298 (+ 230us) server_negotiation.cc:407] Connection header received
0926 16:32:11.632413 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:11.632420 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:11.632614 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:11.632766 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:11.632957 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.633659 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.635232 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.636066 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.636183 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:11.636357 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:11.637578 (+ 1221us) server_negotiation.cc:299] Negotiation successful
0926 16:32:11.637666 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":85,"threads_started":1}
I20260926 16:32:11.639704 12832 tablet_service.cc:1467] Processing CreateTablet for tablet b491740b800741a5844b86a12dd6badc (DEFAULT_TABLE table=test-table [id=8006f20a130745eb8edb50057ef577e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:11.639864 12692 tablet_service.cc:1467] Processing CreateTablet for tablet b491740b800741a5844b86a12dd6badc (DEFAULT_TABLE table=test-table [id=8006f20a130745eb8edb50057ef577e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:11.640308 12552 tablet_service.cc:1467] Processing CreateTablet for tablet b491740b800741a5844b86a12dd6badc (DEFAULT_TABLE table=test-table [id=8006f20a130745eb8edb50057ef577e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:11.640686 12832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b491740b800741a5844b86a12dd6badc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:11.640861 12692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b491740b800741a5844b86a12dd6badc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:11.641196 12552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b491740b800741a5844b86a12dd6badc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:11.647176 12921 tablet_bootstrap.cc:492] T b491740b800741a5844b86a12dd6badc P 4ff1455b40274737b068e75621fc0fb9: Bootstrap starting.
I20260926 16:32:11.648450 12922 tablet_bootstrap.cc:492] T b491740b800741a5844b86a12dd6badc P af9cfd84a2ad48f6864a099828976e6a: Bootstrap starting.
I20260926 16:32:11.649243 12923 tablet_bootstrap.cc:492] T b491740b800741a5844b86a12dd6badc P c84b627d26e04df5b0550e4f520f764a: Bootstrap starting.
I20260926 16:32:11.649926 12921 tablet_bootstrap.cc:654] T b491740b800741a5844b86a12dd6badc P 4ff1455b40274737b068e75621fc0fb9: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:11.650666 12922 tablet_bootstrap.cc:654] T b491740b800741a5844b86a12dd6badc P af9cfd84a2ad48f6864a099828976e6a: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:11.650869 12921 log.cc:824] T b491740b800741a5844b86a12dd6badc P 4ff1455b40274737b068e75621fc0fb9: Log is configured to *not* fsync() on all Append() calls
I20260926 16:32:11.651249 12923 tablet_bootstrap.cc:654] T b491740b800741a5844b86a12dd6badc P c84b627d26e04df5b0550e4f520f764a: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:11.651494 12922 log.cc:824] T b491740b800741a5844b86a12dd6badc P af9cfd84a2ad48f6864a099828976e6a: Log is configured to *not* fsync() on all Append() calls
I20260926 16:32:11.652100 12923 log.cc:824] T b491740b800741a5844b86a12dd6badc P c84b627d26e04df5b0550e4f520f764a: Log is configured to *not* fsync() on all Append() calls
I20260926 16:32:11.653508 12921 tablet_bootstrap.cc:492] T b491740b800741a5844b86a12dd6badc P 4ff1455b40274737b068e75621fc0fb9: No bootstrap required, opened a new log
I20260926 16:32:11.653587 12922 tablet_bootstrap.cc:492] T b491740b800741a5844b86a12dd6badc P af9cfd84a2ad48f6864a099828976e6a: No bootstrap required, opened a new log
I20260926 16:32:11.653661 12921 ts_tablet_manager.cc:1397] T b491740b800741a5844b86a12dd6badc P 4ff1455b40274737b068e75621fc0fb9: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260926 16:32:11.653764 12922 ts_tablet_manager.cc:1397] T b491740b800741a5844b86a12dd6badc P af9cfd84a2ad48f6864a099828976e6a: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260926 16:32:11.653892 12923 tablet_bootstrap.cc:492] T b491740b800741a5844b86a12dd6badc P c84b627d26e04df5b0550e4f520f764a: No bootstrap required, opened a new log
I20260926 16:32:11.654090 12923 ts_tablet_manager.cc:1397] T b491740b800741a5844b86a12dd6badc P c84b627d26e04df5b0550e4f520f764a: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260926 16:32:11.656718 12921 raft_consensus.cc:348] T b491740b800741a5844b86a12dd6badc P 4ff1455b40274737b068e75621fc0fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 } } peers { permanent_uuid: "4ff1455b40274737b068e75621fc0fb9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33629 } } peers { permanent_uuid: "af9cfd84a2ad48f6864a099828976e6a" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40449 } }
I20260926 16:32:11.657012 12921 raft_consensus.cc:374] T b491740b800741a5844b86a12dd6badc P 4ff1455b40274737b068e75621fc0fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:11.657068 12921 raft_consensus.cc:729] T b491740b800741a5844b86a12dd6badc P 4ff1455b40274737b068e75621fc0fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ff1455b40274737b068e75621fc0fb9, State: Initialized, Role: FOLLOWER
I20260926 16:32:11.657567 12921 consensus_queue.cc:260] T b491740b800741a5844b86a12dd6badc P 4ff1455b40274737b068e75621fc0fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 } } peers { permanent_uuid: "4ff1455b40274737b068e75621fc0fb9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33629 } } peers { permanent_uuid: "af9cfd84a2ad48f6864a099828976e6a" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40449 } }
I20260926 16:32:11.657780 12923 raft_consensus.cc:348] T b491740b800741a5844b86a12dd6badc P c84b627d26e04df5b0550e4f520f764a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 } } peers { permanent_uuid: "4ff1455b40274737b068e75621fc0fb9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33629 } } peers { permanent_uuid: "af9cfd84a2ad48f6864a099828976e6a" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40449 } }
I20260926 16:32:11.657780 12922 raft_consensus.cc:348] T b491740b800741a5844b86a12dd6badc P af9cfd84a2ad48f6864a099828976e6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 } } peers { permanent_uuid: "4ff1455b40274737b068e75621fc0fb9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33629 } } peers { permanent_uuid: "af9cfd84a2ad48f6864a099828976e6a" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40449 } }
I20260926 16:32:11.658042 12923 raft_consensus.cc:374] T b491740b800741a5844b86a12dd6badc P c84b627d26e04df5b0550e4f520f764a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:11.658042 12922 raft_consensus.cc:374] T b491740b800741a5844b86a12dd6badc P af9cfd84a2ad48f6864a099828976e6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:11.658103 12923 raft_consensus.cc:729] T b491740b800741a5844b86a12dd6badc P c84b627d26e04df5b0550e4f520f764a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c84b627d26e04df5b0550e4f520f764a, State: Initialized, Role: FOLLOWER
I20260926 16:32:11.658113 12922 raft_consensus.cc:729] T b491740b800741a5844b86a12dd6badc P af9cfd84a2ad48f6864a099828976e6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af9cfd84a2ad48f6864a099828976e6a, State: Initialized, Role: FOLLOWER
I20260926 16:32:11.658413 12898 heartbeater.cc:503] Master 127.11.8.254:36907 was elected leader, sending a full tablet report...
I20260926 16:32:11.658644 12921 ts_tablet_manager.cc:1428] T b491740b800741a5844b86a12dd6badc P 4ff1455b40274737b068e75621fc0fb9: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260926 16:32:11.658630 12923 consensus_queue.cc:260] T b491740b800741a5844b86a12dd6badc P c84b627d26e04df5b0550e4f520f764a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 } } peers { permanent_uuid: "4ff1455b40274737b068e75621fc0fb9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33629 } } peers { permanent_uuid: "af9cfd84a2ad48f6864a099828976e6a" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40449 } }
I20260926 16:32:11.658694 12922 consensus_queue.cc:260] T b491740b800741a5844b86a12dd6badc P af9cfd84a2ad48f6864a099828976e6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 } } peers { permanent_uuid: "4ff1455b40274737b068e75621fc0fb9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33629 } } peers { permanent_uuid: "af9cfd84a2ad48f6864a099828976e6a" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40449 } }
I20260926 16:32:11.659578 12923 ts_tablet_manager.cc:1428] T b491740b800741a5844b86a12dd6badc P c84b627d26e04df5b0550e4f520f764a: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260926 16:32:11.659578 12922 ts_tablet_manager.cc:1428] T b491740b800741a5844b86a12dd6badc P af9cfd84a2ad48f6864a099828976e6a: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260926 16:32:11.659904 12618 heartbeater.cc:503] Master 127.11.8.254:36907 was elected leader, sending a full tablet report...
I20260926 16:32:11.659904 12758 heartbeater.cc:503] Master 127.11.8.254:36907 was elected leader, sending a full tablet report...
W20260926 16:32:11.830766 12899 tablet.cc:2369] T b491740b800741a5844b86a12dd6badc P 4ff1455b40274737b068e75621fc0fb9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 16:32:11.872447 12759 tablet.cc:2369] T b491740b800741a5844b86a12dd6badc P c84b627d26e04df5b0550e4f520f764a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 16:32:11.905090 12619 tablet.cc:2369] T b491740b800741a5844b86a12dd6badc P af9cfd84a2ad48f6864a099828976e6a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:32:11.979494 12927 raft_consensus.cc:482] T b491740b800741a5844b86a12dd6badc P 4ff1455b40274737b068e75621fc0fb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:32:11.979722 12927 raft_consensus.cc:504] T b491740b800741a5844b86a12dd6badc P 4ff1455b40274737b068e75621fc0fb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 } } peers { permanent_uuid: "4ff1455b40274737b068e75621fc0fb9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33629 } } peers { permanent_uuid: "af9cfd84a2ad48f6864a099828976e6a" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40449 } }
I20260926 16:32:11.980976 12927 leader_election.cc:288] T b491740b800741a5844b86a12dd6badc P 4ff1455b40274737b068e75621fc0fb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c84b627d26e04df5b0550e4f520f764a (127.11.8.194:44717), af9cfd84a2ad48f6864a099828976e6a (127.11.8.193:40449)
I20260926 16:32:11.985113 12791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:11.981200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:44717 (local address 127.11.8.195:46837)
0926 16:32:11.981376 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:11.981405 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:11.981503 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:11.981796 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:11.981799 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:11.981819 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:11.981951 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:11.981960 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.982964 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.982967 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:11.983913 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:11.983927 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.984657 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.984660 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:11.984774 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:11.984801 (+ 27us) client_negotiation.cc:769] Sending connection context
0926 16:32:11.984885 (+ 84us) client_negotiation.cc:240] Negotiation successful
0926 16:32:11.984972 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100}
I20260926 16:32:11.985123 12916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:11.981281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:46837 (local address 127.11.8.194:44717)
0926 16:32:11.981390 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:11.981394 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:11.981547 (+ 153us) server_negotiation.cc:407] Connection header received
0926 16:32:11.981616 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:11.981620 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:11.981690 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:11.981771 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:11.982068 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.982855 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.984053 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.984560 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.984645 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:11.984686 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:11.984904 (+ 218us) server_negotiation.cc:299] Negotiation successful
0926 16:32:11.984997 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 16:32:11.985610 12930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:11.981269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:40449 (local address 127.11.8.195:42097)
0926 16:32:11.981616 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:11.981630 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:11.981741 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:11.982035 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:11.982038 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:11.982055 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:11.982520 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:11.982530 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.983345 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.983348 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:11.984239 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:11.984248 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.985066 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.985070 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:11.985320 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:11.985345 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 16:32:11.985423 (+ 78us) client_negotiation.cc:240] Negotiation successful
0926 16:32:11.985503 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":113,"threads_started":1}
I20260926 16:32:11.985639 12920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:11.981384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:42097 (local address 127.11.8.193:40449)
0926 16:32:11.981508 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:11.981512 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:11.981722 (+ 210us) server_negotiation.cc:407] Connection header received
0926 16:32:11.981827 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:11.981829 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:11.981878 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:11.981998 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:11.982629 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.983230 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.984373 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:11.984880 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:11.985067 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:11.985185 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:11.985456 (+ 271us) server_negotiation.cc:299] Negotiation successful
0926 16:32:11.985533 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 16:32:11.985731 12712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b491740b800741a5844b86a12dd6badc" candidate_uuid: "4ff1455b40274737b068e75621fc0fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c84b627d26e04df5b0550e4f520f764a" is_pre_election: true
I20260926 16:32:11.986038 12712 raft_consensus.cc:2454] T b491740b800741a5844b86a12dd6badc P c84b627d26e04df5b0550e4f520f764a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ff1455b40274737b068e75621fc0fb9 in term 0.
I20260926 16:32:11.986268 12572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b491740b800741a5844b86a12dd6badc" candidate_uuid: "4ff1455b40274737b068e75621fc0fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af9cfd84a2ad48f6864a099828976e6a" is_pre_election: true
I20260926 16:32:11.986543 12786 leader_election.cc:302] T b491740b800741a5844b86a12dd6badc P 4ff1455b40274737b068e75621fc0fb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ff1455b40274737b068e75621fc0fb9, c84b627d26e04df5b0550e4f520f764a; no voters:
I20260926 16:32:11.986586 12572 raft_consensus.cc:2454] T b491740b800741a5844b86a12dd6badc P af9cfd84a2ad48f6864a099828976e6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ff1455b40274737b068e75621fc0fb9 in term 0.
I20260926 16:32:11.986799 12927 raft_consensus.cc:2781] T b491740b800741a5844b86a12dd6badc P 4ff1455b40274737b068e75621fc0fb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:32:11.986900 12927 raft_consensus.cc:482] T b491740b800741a5844b86a12dd6badc P 4ff1455b40274737b068e75621fc0fb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:32:11.986965 12927 raft_consensus.cc:3037] T b491740b800741a5844b86a12dd6badc P 4ff1455b40274737b068e75621fc0fb9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:11.988001 12927 raft_consensus.cc:504] T b491740b800741a5844b86a12dd6badc P 4ff1455b40274737b068e75621fc0fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 } } peers { permanent_uuid: "4ff1455b40274737b068e75621fc0fb9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33629 } } peers { permanent_uuid: "af9cfd84a2ad48f6864a099828976e6a" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40449 } }
I20260926 16:32:11.988328 12927 leader_election.cc:288] T b491740b800741a5844b86a12dd6badc P 4ff1455b40274737b068e75621fc0fb9 [CANDIDATE]: Term 1 election: Requested vote from peers c84b627d26e04df5b0550e4f520f764a (127.11.8.194:44717), af9cfd84a2ad48f6864a099828976e6a (127.11.8.193:40449)
I20260926 16:32:11.988659 12712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b491740b800741a5844b86a12dd6badc" candidate_uuid: "4ff1455b40274737b068e75621fc0fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c84b627d26e04df5b0550e4f520f764a"
I20260926 16:32:11.988672 12572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b491740b800741a5844b86a12dd6badc" candidate_uuid: "4ff1455b40274737b068e75621fc0fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af9cfd84a2ad48f6864a099828976e6a"
I20260926 16:32:11.988780 12712 raft_consensus.cc:3037] T b491740b800741a5844b86a12dd6badc P c84b627d26e04df5b0550e4f520f764a [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:11.988794 12572 raft_consensus.cc:3037] T b491740b800741a5844b86a12dd6badc P af9cfd84a2ad48f6864a099828976e6a [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:11.990012 12712 raft_consensus.cc:2454] T b491740b800741a5844b86a12dd6badc P c84b627d26e04df5b0550e4f520f764a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ff1455b40274737b068e75621fc0fb9 in term 1.
I20260926 16:32:11.990077 12572 raft_consensus.cc:2454] T b491740b800741a5844b86a12dd6badc P af9cfd84a2ad48f6864a099828976e6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ff1455b40274737b068e75621fc0fb9 in term 1.
I20260926 16:32:11.990341 12786 leader_election.cc:302] T b491740b800741a5844b86a12dd6badc P 4ff1455b40274737b068e75621fc0fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ff1455b40274737b068e75621fc0fb9, c84b627d26e04df5b0550e4f520f764a; no voters:
I20260926 16:32:11.990513 12927 raft_consensus.cc:2781] T b491740b800741a5844b86a12dd6badc P 4ff1455b40274737b068e75621fc0fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:32:11.990680 12927 raft_consensus.cc:686] T b491740b800741a5844b86a12dd6badc P 4ff1455b40274737b068e75621fc0fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 4ff1455b40274737b068e75621fc0fb9, State: Running, Role: LEADER
I20260926 16:32:11.990988 12927 consensus_queue.cc:237] T b491740b800741a5844b86a12dd6badc P 4ff1455b40274737b068e75621fc0fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 } } peers { permanent_uuid: "4ff1455b40274737b068e75621fc0fb9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33629 } } peers { permanent_uuid: "af9cfd84a2ad48f6864a099828976e6a" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40449 } }
I20260926 16:32:11.993983 12414 catalog_manager.cc:5495] T b491740b800741a5844b86a12dd6badc P 4ff1455b40274737b068e75621fc0fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ff1455b40274737b068e75621fc0fb9 (127.11.8.195). New cstate: current_term: 1 leader_uuid: "4ff1455b40274737b068e75621fc0fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ff1455b40274737b068e75621fc0fb9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33629 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af9cfd84a2ad48f6864a099828976e6a" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40449 } health_report { overall_health: UNKNOWN } } }
I20260926 16:32:12.090874 12917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:12.086968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33380 (local address 127.11.8.195:33629)
0926 16:32:12.087118 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:12.087124 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:12.087211 (+ 87us) server_negotiation.cc:407] Connection header received
0926 16:32:12.087449 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:12.087453 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:12.087518 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:12.087626 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:32:12.087878 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:12.088393 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:12.089296 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:12.089536 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:12.089652 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:12.089996 (+ 344us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:32:12.090442 (+ 446us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:32:12.090572 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:12.090658 (+ 86us) server_negotiation.cc:299] Negotiation successful
0926 16:32:12.090722 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 16:32:12.098203 12572 raft_consensus.cc:1261] T b491740b800741a5844b86a12dd6badc P af9cfd84a2ad48f6864a099828976e6a [term 1 FOLLOWER]: Refusing update from remote peer 4ff1455b40274737b068e75621fc0fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:32:12.098356 12712 raft_consensus.cc:1261] T b491740b800741a5844b86a12dd6badc P c84b627d26e04df5b0550e4f520f764a [term 1 FOLLOWER]: Refusing update from remote peer 4ff1455b40274737b068e75621fc0fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:32:12.099002 12927 consensus_queue.cc:1035] T b491740b800741a5844b86a12dd6badc P 4ff1455b40274737b068e75621fc0fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:32:12.099190 12931 consensus_queue.cc:1035] T b491740b800741a5844b86a12dd6badc P 4ff1455b40274737b068e75621fc0fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af9cfd84a2ad48f6864a099828976e6a" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:32:12.107268 12935 mvcc.cc:205] Tried to move back new op lower bound from 7333643600266170368 to 7333643599837773824. Current Snapshot: MvccSnapshot[applied={T|T < 7333643600266170368}]
I20260926 16:32:12.116984 12415 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47068:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:32:12.117162 12415 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47068:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:32:12.119603 12415 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5303c2fae4bb4b36befb691bd3c73723: Sending DeleteTablet for 3 replicas of tablet b491740b800741a5844b86a12dd6badc
I20260926 16:32:12.120309 12832 tablet_service.cc:1514] Processing DeleteTablet for tablet b491740b800741a5844b86a12dd6badc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:32:12 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:33372
I20260926 16:32:12.120311 12692 tablet_service.cc:1514] Processing DeleteTablet for tablet b491740b800741a5844b86a12dd6badc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:32:12 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:42488
I20260926 16:32:12.120507 12552 tablet_service.cc:1514] Processing DeleteTablet for tablet b491740b800741a5844b86a12dd6badc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:32:12 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:50666
I20260926 16:32:12.120782 12944 tablet_replica.cc:330] T b491740b800741a5844b86a12dd6badc P 4ff1455b40274737b068e75621fc0fb9: stopping tablet replica
I20260926 16:32:12.120786 12943 tablet_replica.cc:330] T b491740b800741a5844b86a12dd6badc P c84b627d26e04df5b0550e4f520f764a: stopping tablet replica
I20260926 16:32:12.121078 12944 raft_consensus.cc:2229] T b491740b800741a5844b86a12dd6badc P 4ff1455b40274737b068e75621fc0fb9 [term 1 LEADER]: Raft consensus shutting down.
I20260926 16:32:12.121119 12943 raft_consensus.cc:2229] T b491740b800741a5844b86a12dd6badc P c84b627d26e04df5b0550e4f520f764a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:32:12.121433 12944 raft_consensus.cc:2258] T b491740b800741a5844b86a12dd6badc P 4ff1455b40274737b068e75621fc0fb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:32:12.121603 12943 raft_consensus.cc:2258] T b491740b800741a5844b86a12dd6badc P c84b627d26e04df5b0550e4f520f764a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:32:12.121591 12415 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47068:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:32:12.122020 12415 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:32:12.122786 12944 ts_tablet_manager.cc:1905] T b491740b800741a5844b86a12dd6badc P 4ff1455b40274737b068e75621fc0fb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:32:12.123303 12943 ts_tablet_manager.cc:1905] T b491740b800741a5844b86a12dd6badc P c84b627d26e04df5b0550e4f520f764a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:32:12.124011 12945 tablet_replica.cc:330] T b491740b800741a5844b86a12dd6badc P af9cfd84a2ad48f6864a099828976e6a: stopping tablet replica
I20260926 16:32:12.124279 12945 raft_consensus.cc:2229] T b491740b800741a5844b86a12dd6badc P af9cfd84a2ad48f6864a099828976e6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:32:12.124563 12945 raft_consensus.cc:2258] T b491740b800741a5844b86a12dd6badc P af9cfd84a2ad48f6864a099828976e6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:32:12.125828 12945 ts_tablet_manager.cc:1905] T b491740b800741a5844b86a12dd6badc P af9cfd84a2ad48f6864a099828976e6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:32:12.126451 12943 ts_tablet_manager.cc:1918] T b491740b800741a5844b86a12dd6badc P c84b627d26e04df5b0550e4f520f764a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:32:12.126587 12943 log.cc:1196] T b491740b800741a5844b86a12dd6badc P c84b627d26e04df5b0550e4f520f764a: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-1/wal/wals/b491740b800741a5844b86a12dd6badc
I20260926 16:32:12.126963 12943 ts_tablet_manager.cc:1939] T b491740b800741a5844b86a12dd6badc P c84b627d26e04df5b0550e4f520f764a: Deleting consensus metadata
I20260926 16:32:12.128607 12692 tablet_service.cc:1467] Processing CreateTablet for tablet 6965e80d53af47ab99e1c5f348f1ed4a (DEFAULT_TABLE table=test-table [id=76cbfd4011754caea499838bc678b005]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:12.128758 12945 ts_tablet_manager.cc:1918] T b491740b800741a5844b86a12dd6badc P af9cfd84a2ad48f6864a099828976e6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:32:12.128855 12945 log.cc:1196] T b491740b800741a5844b86a12dd6badc P af9cfd84a2ad48f6864a099828976e6a: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-0/wal/wals/b491740b800741a5844b86a12dd6badc
I20260926 16:32:12.128854 12692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6965e80d53af47ab99e1c5f348f1ed4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:12.129154 12945 ts_tablet_manager.cc:1939] T b491740b800741a5844b86a12dd6badc P af9cfd84a2ad48f6864a099828976e6a: Deleting consensus metadata
I20260926 16:32:12.130546 12832 tablet_service.cc:1467] Processing CreateTablet for tablet 6965e80d53af47ab99e1c5f348f1ed4a (DEFAULT_TABLE table=test-table [id=76cbfd4011754caea499838bc678b005]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:12.130834 12832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6965e80d53af47ab99e1c5f348f1ed4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:12.130980 12923 tablet_bootstrap.cc:492] T 6965e80d53af47ab99e1c5f348f1ed4a P c84b627d26e04df5b0550e4f520f764a: Bootstrap starting.
I20260926 16:32:12.131991 12923 tablet_bootstrap.cc:654] T 6965e80d53af47ab99e1c5f348f1ed4a P c84b627d26e04df5b0550e4f520f764a: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:12.132932 12402 catalog_manager.cc:4841] TS c84b627d26e04df5b0550e4f520f764a (127.11.8.194:44717): tablet b491740b800741a5844b86a12dd6badc (table test-table [id=8006f20a130745eb8edb50057ef577e0]) successfully deleted
I20260926 16:32:12.133549 12921 tablet_bootstrap.cc:492] T 6965e80d53af47ab99e1c5f348f1ed4a P 4ff1455b40274737b068e75621fc0fb9: Bootstrap starting.
I20260926 16:32:12.133536 12399 catalog_manager.cc:4841] TS af9cfd84a2ad48f6864a099828976e6a (127.11.8.193:40449): tablet b491740b800741a5844b86a12dd6badc (table test-table [id=8006f20a130745eb8edb50057ef577e0]) successfully deleted
I20260926 16:32:12.133769 12923 tablet_bootstrap.cc:492] T 6965e80d53af47ab99e1c5f348f1ed4a P c84b627d26e04df5b0550e4f520f764a: No bootstrap required, opened a new log
I20260926 16:32:12.133877 12923 ts_tablet_manager.cc:1397] T 6965e80d53af47ab99e1c5f348f1ed4a P c84b627d26e04df5b0550e4f520f764a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260926 16:32:12.134058 12552 tablet_service.cc:1467] Processing CreateTablet for tablet 6965e80d53af47ab99e1c5f348f1ed4a (DEFAULT_TABLE table=test-table [id=76cbfd4011754caea499838bc678b005]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:12.134344 12552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6965e80d53af47ab99e1c5f348f1ed4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:12.134364 12923 raft_consensus.cc:348] T 6965e80d53af47ab99e1c5f348f1ed4a P c84b627d26e04df5b0550e4f520f764a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af9cfd84a2ad48f6864a099828976e6a" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40449 } } peers { permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 } } peers { permanent_uuid: "4ff1455b40274737b068e75621fc0fb9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33629 } }
I20260926 16:32:12.134491 12923 raft_consensus.cc:374] T 6965e80d53af47ab99e1c5f348f1ed4a P c84b627d26e04df5b0550e4f520f764a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:12.134511 12921 tablet_bootstrap.cc:654] T 6965e80d53af47ab99e1c5f348f1ed4a P 4ff1455b40274737b068e75621fc0fb9: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:12.134529 12923 raft_consensus.cc:729] T 6965e80d53af47ab99e1c5f348f1ed4a P c84b627d26e04df5b0550e4f520f764a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c84b627d26e04df5b0550e4f520f764a, State: Initialized, Role: FOLLOWER
I20260926 16:32:12.134673 12923 consensus_queue.cc:260] T 6965e80d53af47ab99e1c5f348f1ed4a P c84b627d26e04df5b0550e4f520f764a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af9cfd84a2ad48f6864a099828976e6a" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40449 } } peers { permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 } } peers { permanent_uuid: "4ff1455b40274737b068e75621fc0fb9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33629 } }
I20260926 16:32:12.134938 12923 ts_tablet_manager.cc:1428] T 6965e80d53af47ab99e1c5f348f1ed4a P c84b627d26e04df5b0550e4f520f764a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 16:32:12.135703 12921 tablet_bootstrap.cc:492] T 6965e80d53af47ab99e1c5f348f1ed4a P 4ff1455b40274737b068e75621fc0fb9: No bootstrap required, opened a new log
I20260926 16:32:12.135784 12921 ts_tablet_manager.cc:1397] T 6965e80d53af47ab99e1c5f348f1ed4a P 4ff1455b40274737b068e75621fc0fb9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 16:32:12.136227 12921 raft_consensus.cc:348] T 6965e80d53af47ab99e1c5f348f1ed4a P 4ff1455b40274737b068e75621fc0fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af9cfd84a2ad48f6864a099828976e6a" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40449 } } peers { permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 } } peers { permanent_uuid: "4ff1455b40274737b068e75621fc0fb9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33629 } }
I20260926 16:32:12.136341 12921 raft_consensus.cc:374] T 6965e80d53af47ab99e1c5f348f1ed4a P 4ff1455b40274737b068e75621fc0fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:12.136370 12921 raft_consensus.cc:729] T 6965e80d53af47ab99e1c5f348f1ed4a P 4ff1455b40274737b068e75621fc0fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ff1455b40274737b068e75621fc0fb9, State: Initialized, Role: FOLLOWER
I20260926 16:32:12.136495 12921 consensus_queue.cc:260] T 6965e80d53af47ab99e1c5f348f1ed4a P 4ff1455b40274737b068e75621fc0fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af9cfd84a2ad48f6864a099828976e6a" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40449 } } peers { permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 } } peers { permanent_uuid: "4ff1455b40274737b068e75621fc0fb9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33629 } }
I20260926 16:32:12.136744 12921 ts_tablet_manager.cc:1428] T 6965e80d53af47ab99e1c5f348f1ed4a P 4ff1455b40274737b068e75621fc0fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 16:32:12.137038 12922 tablet_bootstrap.cc:492] T 6965e80d53af47ab99e1c5f348f1ed4a P af9cfd84a2ad48f6864a099828976e6a: Bootstrap starting.
I20260926 16:32:12.138018 12922 tablet_bootstrap.cc:654] T 6965e80d53af47ab99e1c5f348f1ed4a P af9cfd84a2ad48f6864a099828976e6a: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:12.139374 12922 tablet_bootstrap.cc:492] T 6965e80d53af47ab99e1c5f348f1ed4a P af9cfd84a2ad48f6864a099828976e6a: No bootstrap required, opened a new log
I20260926 16:32:12.139453 12922 ts_tablet_manager.cc:1397] T 6965e80d53af47ab99e1c5f348f1ed4a P af9cfd84a2ad48f6864a099828976e6a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 16:32:12.140272 12922 raft_consensus.cc:348] T 6965e80d53af47ab99e1c5f348f1ed4a P af9cfd84a2ad48f6864a099828976e6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af9cfd84a2ad48f6864a099828976e6a" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40449 } } peers { permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 } } peers { permanent_uuid: "4ff1455b40274737b068e75621fc0fb9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33629 } }
I20260926 16:32:12.140388 12922 raft_consensus.cc:374] T 6965e80d53af47ab99e1c5f348f1ed4a P af9cfd84a2ad48f6864a099828976e6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:12.140436 12922 raft_consensus.cc:729] T 6965e80d53af47ab99e1c5f348f1ed4a P af9cfd84a2ad48f6864a099828976e6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af9cfd84a2ad48f6864a099828976e6a, State: Initialized, Role: FOLLOWER
I20260926 16:32:12.140578 12922 consensus_queue.cc:260] T 6965e80d53af47ab99e1c5f348f1ed4a P af9cfd84a2ad48f6864a099828976e6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af9cfd84a2ad48f6864a099828976e6a" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40449 } } peers { permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 } } peers { permanent_uuid: "4ff1455b40274737b068e75621fc0fb9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33629 } }
I20260926 16:32:12.140640 12944 ts_tablet_manager.cc:1918] T b491740b800741a5844b86a12dd6badc P 4ff1455b40274737b068e75621fc0fb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:32:12.140726 12944 log.cc:1196] T b491740b800741a5844b86a12dd6badc P 4ff1455b40274737b068e75621fc0fb9: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-2/wal/wals/b491740b800741a5844b86a12dd6badc
I20260926 16:32:12.140844 12922 ts_tablet_manager.cc:1428] T 6965e80d53af47ab99e1c5f348f1ed4a P af9cfd84a2ad48f6864a099828976e6a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260926 16:32:12.141075 12944 ts_tablet_manager.cc:1939] T b491740b800741a5844b86a12dd6badc P 4ff1455b40274737b068e75621fc0fb9: Deleting consensus metadata
I20260926 16:32:12.142211 12401 catalog_manager.cc:4841] TS 4ff1455b40274737b068e75621fc0fb9 (127.11.8.195:33629): tablet b491740b800741a5844b86a12dd6badc (table test-table [id=8006f20a130745eb8edb50057ef577e0]) successfully deleted
I20260926 16:32:12.243276 12928 raft_consensus.cc:482] T 6965e80d53af47ab99e1c5f348f1ed4a P af9cfd84a2ad48f6864a099828976e6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:32:12.243428 12928 raft_consensus.cc:504] T 6965e80d53af47ab99e1c5f348f1ed4a P af9cfd84a2ad48f6864a099828976e6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af9cfd84a2ad48f6864a099828976e6a" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40449 } } peers { permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 } } peers { permanent_uuid: "4ff1455b40274737b068e75621fc0fb9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33629 } }
I20260926 16:32:12.244505 12928 leader_election.cc:288] T 6965e80d53af47ab99e1c5f348f1ed4a P af9cfd84a2ad48f6864a099828976e6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c84b627d26e04df5b0550e4f520f764a (127.11.8.194:44717), 4ff1455b40274737b068e75621fc0fb9 (127.11.8.195:33629)
I20260926 16:32:12.249104 12949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:12.244670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:44717 (local address 127.11.8.193:55037)
0926 16:32:12.245011 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:12.245027 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:12.245186 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:12.245578 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:12.245581 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:12.245603 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:12.245723 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:12.245730 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:12.246818 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:12.246821 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:12.247918 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:12.247945 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:12.248641 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:12.248645 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:12.248752 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:12.248780 (+ 28us) client_negotiation.cc:769] Sending connection context
0926 16:32:12.248868 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 16:32:12.248947 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":78,"thread_start_us":98,"threads_started":1}
I20260926 16:32:12.249138 12916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:12.244813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:55037 (local address 127.11.8.194:44717)
0926 16:32:12.244961 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:12.244965 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:12.245101 (+ 136us) server_negotiation.cc:407] Connection header received
0926 16:32:12.245322 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:12.245328 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:12.245387 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:12.245434 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:12.245889 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:12.246688 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:12.248064 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:12.248512 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:12.248626 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:12.248679 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:12.248897 (+ 218us) server_negotiation.cc:299] Negotiation successful
0926 16:32:12.248973 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 16:32:12.249698 12712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6965e80d53af47ab99e1c5f348f1ed4a" candidate_uuid: "af9cfd84a2ad48f6864a099828976e6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c84b627d26e04df5b0550e4f520f764a" is_pre_election: true
I20260926 16:32:12.249852 12712 raft_consensus.cc:2454] T 6965e80d53af47ab99e1c5f348f1ed4a P c84b627d26e04df5b0550e4f520f764a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af9cfd84a2ad48f6864a099828976e6a in term 0.
I20260926 16:32:12.249900 12917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:12.245173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:41667 (local address 127.11.8.195:33629)
0926 16:32:12.245293 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:12.245297 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:12.245312 (+ 15us) server_negotiation.cc:407] Connection header received
0926 16:32:12.245367 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:12.245370 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:12.245418 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:12.245510 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:12.246406 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:12.246954 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:12.248456 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:12.248996 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:12.249109 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:12.249158 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:12.249715 (+ 557us) server_negotiation.cc:299] Negotiation successful
0926 16:32:12.249790 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 16:32:12.249915 12950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:12.244780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:33629 (local address 127.11.8.193:41667)
0926 16:32:12.245153 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:12.245165 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:12.245239 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:12.245545 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:12.245548 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:12.245570 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:12.245735 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:12.245740 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:12.247098 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:12.247102 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:12.248310 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:12.248322 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:12.249467 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:12.249471 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:12.249572 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:12.249596 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 16:32:12.249685 (+ 89us) client_negotiation.cc:240] Negotiation successful
0926 16:32:12.249748 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"mutex_wait_us":22,"spinlock_wait_cycles":768,"thread_start_us":109,"threads_started":1}
I20260926 16:32:12.250379 12852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6965e80d53af47ab99e1c5f348f1ed4a" candidate_uuid: "af9cfd84a2ad48f6864a099828976e6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ff1455b40274737b068e75621fc0fb9" is_pre_election: true
I20260926 16:32:12.250402 12506 leader_election.cc:302] T 6965e80d53af47ab99e1c5f348f1ed4a P af9cfd84a2ad48f6864a099828976e6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af9cfd84a2ad48f6864a099828976e6a, c84b627d26e04df5b0550e4f520f764a; no voters:
I20260926 16:32:12.250605 12852 raft_consensus.cc:2454] T 6965e80d53af47ab99e1c5f348f1ed4a P 4ff1455b40274737b068e75621fc0fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af9cfd84a2ad48f6864a099828976e6a in term 0.
I20260926 16:32:12.250651 12928 raft_consensus.cc:2781] T 6965e80d53af47ab99e1c5f348f1ed4a P af9cfd84a2ad48f6864a099828976e6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:32:12.250705 12928 raft_consensus.cc:482] T 6965e80d53af47ab99e1c5f348f1ed4a P af9cfd84a2ad48f6864a099828976e6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:32:12.250749 12928 raft_consensus.cc:3037] T 6965e80d53af47ab99e1c5f348f1ed4a P af9cfd84a2ad48f6864a099828976e6a [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:12.251906 12928 raft_consensus.cc:504] T 6965e80d53af47ab99e1c5f348f1ed4a P af9cfd84a2ad48f6864a099828976e6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af9cfd84a2ad48f6864a099828976e6a" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40449 } } peers { permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 } } peers { permanent_uuid: "4ff1455b40274737b068e75621fc0fb9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33629 } }
I20260926 16:32:12.252349 12928 leader_election.cc:288] T 6965e80d53af47ab99e1c5f348f1ed4a P af9cfd84a2ad48f6864a099828976e6a [CANDIDATE]: Term 1 election: Requested vote from peers c84b627d26e04df5b0550e4f520f764a (127.11.8.194:44717), 4ff1455b40274737b068e75621fc0fb9 (127.11.8.195:33629)
I20260926 16:32:12.252746 12852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6965e80d53af47ab99e1c5f348f1ed4a" candidate_uuid: "af9cfd84a2ad48f6864a099828976e6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ff1455b40274737b068e75621fc0fb9"
I20260926 16:32:12.252748 12712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6965e80d53af47ab99e1c5f348f1ed4a" candidate_uuid: "af9cfd84a2ad48f6864a099828976e6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c84b627d26e04df5b0550e4f520f764a"
I20260926 16:32:12.252876 12852 raft_consensus.cc:3037] T 6965e80d53af47ab99e1c5f348f1ed4a P 4ff1455b40274737b068e75621fc0fb9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:12.252876 12712 raft_consensus.cc:3037] T 6965e80d53af47ab99e1c5f348f1ed4a P c84b627d26e04df5b0550e4f520f764a [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:12.254031 12852 raft_consensus.cc:2454] T 6965e80d53af47ab99e1c5f348f1ed4a P 4ff1455b40274737b068e75621fc0fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af9cfd84a2ad48f6864a099828976e6a in term 1.
I20260926 16:32:12.254031 12712 raft_consensus.cc:2454] T 6965e80d53af47ab99e1c5f348f1ed4a P c84b627d26e04df5b0550e4f520f764a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af9cfd84a2ad48f6864a099828976e6a in term 1.
I20260926 16:32:12.254354 12505 leader_election.cc:302] T 6965e80d53af47ab99e1c5f348f1ed4a P af9cfd84a2ad48f6864a099828976e6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ff1455b40274737b068e75621fc0fb9, af9cfd84a2ad48f6864a099828976e6a; no voters:
I20260926 16:32:12.254544 12928 raft_consensus.cc:2781] T 6965e80d53af47ab99e1c5f348f1ed4a P af9cfd84a2ad48f6864a099828976e6a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:32:12.254801 12928 raft_consensus.cc:686] T 6965e80d53af47ab99e1c5f348f1ed4a P af9cfd84a2ad48f6864a099828976e6a [term 1 LEADER]: Becoming Leader. State: Replica: af9cfd84a2ad48f6864a099828976e6a, State: Running, Role: LEADER
I20260926 16:32:12.255077 12928 consensus_queue.cc:237] T 6965e80d53af47ab99e1c5f348f1ed4a P af9cfd84a2ad48f6864a099828976e6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af9cfd84a2ad48f6864a099828976e6a" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40449 } } peers { permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 } } peers { permanent_uuid: "4ff1455b40274737b068e75621fc0fb9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33629 } }
I20260926 16:32:12.257620 12414 catalog_manager.cc:5495] T 6965e80d53af47ab99e1c5f348f1ed4a P af9cfd84a2ad48f6864a099828976e6a reported cstate change: term changed from 0 to 1, leader changed from <none> to af9cfd84a2ad48f6864a099828976e6a (127.11.8.193). New cstate: current_term: 1 leader_uuid: "af9cfd84a2ad48f6864a099828976e6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af9cfd84a2ad48f6864a099828976e6a" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ff1455b40274737b068e75621fc0fb9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33629 } health_report { overall_health: UNKNOWN } } }
Sep 26 16:32:12 dist-test-slave-2zvq krb5kdc[12370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440330, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:32:12.319931 12959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:12.311880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36907 (local address 127.0.0.1:47072)
0926 16:32:12.312250 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:12.312265 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:12.312386 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:12.312642 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:12.312644 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:12.313050 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:12.313146 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:12.313152 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:12.313904 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:12.313907 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:12.314546 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:12.314555 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:12.314987 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:12.314989 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:12.315048 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:12.315569 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:12.315580 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:12.316926 (+ 1346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:12.318528 (+ 1602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:12.318536 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:12.318541 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:12.318851 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:12.319087 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:12.319089 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:12.319090 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:12.319167 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:12.319475 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:12.319480 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:12.319574 (+ 94us) client_negotiation.cc:769] Sending connection context
0926 16:32:12.319662 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 16:32:12.319773 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":152,"threads_started":1}
I20260926 16:32:12.319968 12958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:12.311927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47072 (local address 127.11.8.254:36907)
0926 16:32:12.312209 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:12.312218 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:12.312381 (+ 163us) server_negotiation.cc:407] Connection header received
0926 16:32:12.312476 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:12.312478 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:12.312520 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:12.312605 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:12.313271 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:12.313807 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:12.314646 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:12.314867 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:12.314969 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:12.317048 (+ 2079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:12.317050 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:12.317052 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:12.317072 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:12.318437 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:12.318933 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:12.318936 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:12.318937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:12.319011 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:12.319239 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:12.319241 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:12.319243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:12.319392 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:12.319464 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:12.319713 (+ 249us) server_negotiation.cc:299] Negotiation successful
0926 16:32:12.319829 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":209,"spinlock_wait_cycles":10496,"thread_start_us":120,"threads_started":1}
I20260926 16:32:12.323544 12414 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:47072:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 16:32:12.324191 12414 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:32:12.328718 12552 tablet_service.cc:1467] Processing CreateTablet for tablet 43cb3454ab7848c18a217d0832ed9c64 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2ab366b852144db4bb3e26baeab6dc10]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 16:32:12.328871 12692 tablet_service.cc:1467] Processing CreateTablet for tablet 43cb3454ab7848c18a217d0832ed9c64 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2ab366b852144db4bb3e26baeab6dc10]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 16:32:12.329150 12552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43cb3454ab7848c18a217d0832ed9c64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:12.329020 12832 tablet_service.cc:1467] Processing CreateTablet for tablet 43cb3454ab7848c18a217d0832ed9c64 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2ab366b852144db4bb3e26baeab6dc10]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 16:32:12.329197 12692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43cb3454ab7848c18a217d0832ed9c64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:12.329396 12832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43cb3454ab7848c18a217d0832ed9c64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:12.332129 12921 tablet_bootstrap.cc:492] T 43cb3454ab7848c18a217d0832ed9c64 P 4ff1455b40274737b068e75621fc0fb9: Bootstrap starting.
I20260926 16:32:12.332242 12923 tablet_bootstrap.cc:492] T 43cb3454ab7848c18a217d0832ed9c64 P c84b627d26e04df5b0550e4f520f764a: Bootstrap starting.
I20260926 16:32:12.332242 12922 tablet_bootstrap.cc:492] T 43cb3454ab7848c18a217d0832ed9c64 P af9cfd84a2ad48f6864a099828976e6a: Bootstrap starting.
I20260926 16:32:12.333043 12921 tablet_bootstrap.cc:654] T 43cb3454ab7848c18a217d0832ed9c64 P 4ff1455b40274737b068e75621fc0fb9: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:12.333366 12923 tablet_bootstrap.cc:654] T 43cb3454ab7848c18a217d0832ed9c64 P c84b627d26e04df5b0550e4f520f764a: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:12.333380 12922 tablet_bootstrap.cc:654] T 43cb3454ab7848c18a217d0832ed9c64 P af9cfd84a2ad48f6864a099828976e6a: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:12.334786 12923 tablet_bootstrap.cc:492] T 43cb3454ab7848c18a217d0832ed9c64 P c84b627d26e04df5b0550e4f520f764a: No bootstrap required, opened a new log
I20260926 16:32:12.334856 12923 ts_tablet_manager.cc:1397] T 43cb3454ab7848c18a217d0832ed9c64 P c84b627d26e04df5b0550e4f520f764a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260926 16:32:12.335078 12922 tablet_bootstrap.cc:492] T 43cb3454ab7848c18a217d0832ed9c64 P af9cfd84a2ad48f6864a099828976e6a: No bootstrap required, opened a new log
I20260926 16:32:12.335175 12922 ts_tablet_manager.cc:1397] T 43cb3454ab7848c18a217d0832ed9c64 P af9cfd84a2ad48f6864a099828976e6a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260926 16:32:12.335330 12923 raft_consensus.cc:348] T 43cb3454ab7848c18a217d0832ed9c64 P c84b627d26e04df5b0550e4f520f764a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af9cfd84a2ad48f6864a099828976e6a" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40449 } } peers { permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 } } peers { permanent_uuid: "4ff1455b40274737b068e75621fc0fb9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33629 } }
I20260926 16:32:12.335453 12923 raft_consensus.cc:374] T 43cb3454ab7848c18a217d0832ed9c64 P c84b627d26e04df5b0550e4f520f764a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:12.335493 12923 raft_consensus.cc:729] T 43cb3454ab7848c18a217d0832ed9c64 P c84b627d26e04df5b0550e4f520f764a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c84b627d26e04df5b0550e4f520f764a, State: Initialized, Role: FOLLOWER
I20260926 16:32:12.335635 12923 consensus_queue.cc:260] T 43cb3454ab7848c18a217d0832ed9c64 P c84b627d26e04df5b0550e4f520f764a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af9cfd84a2ad48f6864a099828976e6a" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40449 } } peers { permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 } } peers { permanent_uuid: "4ff1455b40274737b068e75621fc0fb9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33629 } }
I20260926 16:32:12.335624 12922 raft_consensus.cc:348] T 43cb3454ab7848c18a217d0832ed9c64 P af9cfd84a2ad48f6864a099828976e6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af9cfd84a2ad48f6864a099828976e6a" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40449 } } peers { permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 } } peers { permanent_uuid: "4ff1455b40274737b068e75621fc0fb9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33629 } }
I20260926 16:32:12.335753 12922 raft_consensus.cc:374] T 43cb3454ab7848c18a217d0832ed9c64 P af9cfd84a2ad48f6864a099828976e6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:12.335798 12922 raft_consensus.cc:729] T 43cb3454ab7848c18a217d0832ed9c64 P af9cfd84a2ad48f6864a099828976e6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af9cfd84a2ad48f6864a099828976e6a, State: Initialized, Role: FOLLOWER
I20260926 16:32:12.335927 12923 ts_tablet_manager.cc:1428] T 43cb3454ab7848c18a217d0832ed9c64 P c84b627d26e04df5b0550e4f520f764a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 16:32:12.335923 12922 consensus_queue.cc:260] T 43cb3454ab7848c18a217d0832ed9c64 P af9cfd84a2ad48f6864a099828976e6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af9cfd84a2ad48f6864a099828976e6a" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40449 } } peers { permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 } } peers { permanent_uuid: "4ff1455b40274737b068e75621fc0fb9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33629 } }
I20260926 16:32:12.336133 12929 tablet_replica.cc:439] T 43cb3454ab7848c18a217d0832ed9c64 P c84b627d26e04df5b0550e4f520f764a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af9cfd84a2ad48f6864a099828976e6a" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40449 } } peers { permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 } } peers { permanent_uuid: "4ff1455b40274737b068e75621fc0fb9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33629 } } }
I20260926 16:32:12.336201 12922 ts_tablet_manager.cc:1428] T 43cb3454ab7848c18a217d0832ed9c64 P af9cfd84a2ad48f6864a099828976e6a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 16:32:12.336251 12929 tablet_replica.cc:442] T 43cb3454ab7848c18a217d0832ed9c64 P c84b627d26e04df5b0550e4f520f764a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 16:32:12.336386 12928 tablet_replica.cc:439] T 43cb3454ab7848c18a217d0832ed9c64 P af9cfd84a2ad48f6864a099828976e6a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af9cfd84a2ad48f6864a099828976e6a" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40449 } } peers { permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 } } peers { permanent_uuid: "4ff1455b40274737b068e75621fc0fb9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33629 } } }
I20260926 16:32:12.336503 12928 tablet_replica.cc:442] T 43cb3454ab7848c18a217d0832ed9c64 P af9cfd84a2ad48f6864a099828976e6a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 16:32:12.337208 12921 tablet_bootstrap.cc:492] T 43cb3454ab7848c18a217d0832ed9c64 P 4ff1455b40274737b068e75621fc0fb9: No bootstrap required, opened a new log
I20260926 16:32:12.337299 12921 ts_tablet_manager.cc:1397] T 43cb3454ab7848c18a217d0832ed9c64 P 4ff1455b40274737b068e75621fc0fb9: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260926 16:32:12.337752 12921 raft_consensus.cc:348] T 43cb3454ab7848c18a217d0832ed9c64 P 4ff1455b40274737b068e75621fc0fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af9cfd84a2ad48f6864a099828976e6a" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40449 } } peers { permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 } } peers { permanent_uuid: "4ff1455b40274737b068e75621fc0fb9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33629 } }
I20260926 16:32:12.337886 12921 raft_consensus.cc:374] T 43cb3454ab7848c18a217d0832ed9c64 P 4ff1455b40274737b068e75621fc0fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:12.337918 12921 raft_consensus.cc:729] T 43cb3454ab7848c18a217d0832ed9c64 P 4ff1455b40274737b068e75621fc0fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ff1455b40274737b068e75621fc0fb9, State: Initialized, Role: FOLLOWER
I20260926 16:32:12.338076 12921 consensus_queue.cc:260] T 43cb3454ab7848c18a217d0832ed9c64 P 4ff1455b40274737b068e75621fc0fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af9cfd84a2ad48f6864a099828976e6a" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40449 } } peers { permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 } } peers { permanent_uuid: "4ff1455b40274737b068e75621fc0fb9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33629 } }
I20260926 16:32:12.338370 12921 ts_tablet_manager.cc:1428] T 43cb3454ab7848c18a217d0832ed9c64 P 4ff1455b40274737b068e75621fc0fb9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 16:32:12.338554 12927 tablet_replica.cc:439] T 43cb3454ab7848c18a217d0832ed9c64 P 4ff1455b40274737b068e75621fc0fb9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af9cfd84a2ad48f6864a099828976e6a" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40449 } } peers { permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 } } peers { permanent_uuid: "4ff1455b40274737b068e75621fc0fb9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33629 } } }
I20260926 16:32:12.338696 12927 tablet_replica.cc:442] T 43cb3454ab7848c18a217d0832ed9c64 P 4ff1455b40274737b068e75621fc0fb9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 16:32:12.463280 12927 raft_consensus.cc:482] T 43cb3454ab7848c18a217d0832ed9c64 P 4ff1455b40274737b068e75621fc0fb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:32:12.463426 12927 raft_consensus.cc:504] T 43cb3454ab7848c18a217d0832ed9c64 P 4ff1455b40274737b068e75621fc0fb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af9cfd84a2ad48f6864a099828976e6a" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40449 } } peers { permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 } } peers { permanent_uuid: "4ff1455b40274737b068e75621fc0fb9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33629 } }
I20260926 16:32:12.463917 12927 leader_election.cc:288] T 43cb3454ab7848c18a217d0832ed9c64 P 4ff1455b40274737b068e75621fc0fb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af9cfd84a2ad48f6864a099828976e6a (127.11.8.193:40449), c84b627d26e04df5b0550e4f520f764a (127.11.8.194:44717)
I20260926 16:32:12.464429 12712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43cb3454ab7848c18a217d0832ed9c64" candidate_uuid: "4ff1455b40274737b068e75621fc0fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c84b627d26e04df5b0550e4f520f764a" is_pre_election: true
I20260926 16:32:12.464448 12572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43cb3454ab7848c18a217d0832ed9c64" candidate_uuid: "4ff1455b40274737b068e75621fc0fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af9cfd84a2ad48f6864a099828976e6a" is_pre_election: true
I20260926 16:32:12.464613 12572 raft_consensus.cc:2454] T 43cb3454ab7848c18a217d0832ed9c64 P af9cfd84a2ad48f6864a099828976e6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ff1455b40274737b068e75621fc0fb9 in term 0.
I20260926 16:32:12.464617 12712 raft_consensus.cc:2454] T 43cb3454ab7848c18a217d0832ed9c64 P c84b627d26e04df5b0550e4f520f764a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ff1455b40274737b068e75621fc0fb9 in term 0.
I20260926 16:32:12.464965 12786 leader_election.cc:302] T 43cb3454ab7848c18a217d0832ed9c64 P 4ff1455b40274737b068e75621fc0fb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ff1455b40274737b068e75621fc0fb9, c84b627d26e04df5b0550e4f520f764a; no voters:
I20260926 16:32:12.465159 12927 raft_consensus.cc:2781] T 43cb3454ab7848c18a217d0832ed9c64 P 4ff1455b40274737b068e75621fc0fb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:32:12.465219 12927 raft_consensus.cc:482] T 43cb3454ab7848c18a217d0832ed9c64 P 4ff1455b40274737b068e75621fc0fb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:32:12.465246 12927 raft_consensus.cc:3037] T 43cb3454ab7848c18a217d0832ed9c64 P 4ff1455b40274737b068e75621fc0fb9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:12.466544 12927 raft_consensus.cc:504] T 43cb3454ab7848c18a217d0832ed9c64 P 4ff1455b40274737b068e75621fc0fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af9cfd84a2ad48f6864a099828976e6a" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40449 } } peers { permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 } } peers { permanent_uuid: "4ff1455b40274737b068e75621fc0fb9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33629 } }
I20260926 16:32:12.466914 12927 leader_election.cc:288] T 43cb3454ab7848c18a217d0832ed9c64 P 4ff1455b40274737b068e75621fc0fb9 [CANDIDATE]: Term 1 election: Requested vote from peers af9cfd84a2ad48f6864a099828976e6a (127.11.8.193:40449), c84b627d26e04df5b0550e4f520f764a (127.11.8.194:44717)
I20260926 16:32:12.467268 12572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43cb3454ab7848c18a217d0832ed9c64" candidate_uuid: "4ff1455b40274737b068e75621fc0fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af9cfd84a2ad48f6864a099828976e6a"
I20260926 16:32:12.467267 12712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43cb3454ab7848c18a217d0832ed9c64" candidate_uuid: "4ff1455b40274737b068e75621fc0fb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c84b627d26e04df5b0550e4f520f764a"
I20260926 16:32:12.467402 12572 raft_consensus.cc:3037] T 43cb3454ab7848c18a217d0832ed9c64 P af9cfd84a2ad48f6864a099828976e6a [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:12.467412 12712 raft_consensus.cc:3037] T 43cb3454ab7848c18a217d0832ed9c64 P c84b627d26e04df5b0550e4f520f764a [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:12.468297 12712 raft_consensus.cc:2454] T 43cb3454ab7848c18a217d0832ed9c64 P c84b627d26e04df5b0550e4f520f764a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ff1455b40274737b068e75621fc0fb9 in term 1.
I20260926 16:32:12.468457 12572 raft_consensus.cc:2454] T 43cb3454ab7848c18a217d0832ed9c64 P af9cfd84a2ad48f6864a099828976e6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ff1455b40274737b068e75621fc0fb9 in term 1.
I20260926 16:32:12.468561 12786 leader_election.cc:302] T 43cb3454ab7848c18a217d0832ed9c64 P 4ff1455b40274737b068e75621fc0fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ff1455b40274737b068e75621fc0fb9, c84b627d26e04df5b0550e4f520f764a; no voters:
I20260926 16:32:12.468717 12927 raft_consensus.cc:2781] T 43cb3454ab7848c18a217d0832ed9c64 P 4ff1455b40274737b068e75621fc0fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:32:12.468816 12927 raft_consensus.cc:686] T 43cb3454ab7848c18a217d0832ed9c64 P 4ff1455b40274737b068e75621fc0fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 4ff1455b40274737b068e75621fc0fb9, State: Running, Role: LEADER
I20260926 16:32:12.468993 12927 consensus_queue.cc:237] T 43cb3454ab7848c18a217d0832ed9c64 P 4ff1455b40274737b068e75621fc0fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af9cfd84a2ad48f6864a099828976e6a" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40449 } } peers { permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 } } peers { permanent_uuid: "4ff1455b40274737b068e75621fc0fb9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33629 } }
I20260926 16:32:12.469789 12931 tablet_replica.cc:439] T 43cb3454ab7848c18a217d0832ed9c64 P 4ff1455b40274737b068e75621fc0fb9: TxnStatusTablet state changed. Reason: New leader 4ff1455b40274737b068e75621fc0fb9. Latest consensus state: current_term: 1 leader_uuid: "4ff1455b40274737b068e75621fc0fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af9cfd84a2ad48f6864a099828976e6a" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40449 } } peers { permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 } } peers { permanent_uuid: "4ff1455b40274737b068e75621fc0fb9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33629 } } }
I20260926 16:32:12.469929 12931 tablet_replica.cc:442] T 43cb3454ab7848c18a217d0832ed9c64 P 4ff1455b40274737b068e75621fc0fb9: This TxnStatusTablet replica's current role is: LEADER
I20260926 16:32:12.470230 12964 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 16:32:12.470449 12964 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 16:32:12.470886 12415 catalog_manager.cc:5495] T 43cb3454ab7848c18a217d0832ed9c64 P 4ff1455b40274737b068e75621fc0fb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ff1455b40274737b068e75621fc0fb9 (127.11.8.195). New cstate: current_term: 1 leader_uuid: "4ff1455b40274737b068e75621fc0fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af9cfd84a2ad48f6864a099828976e6a" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ff1455b40274737b068e75621fc0fb9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33629 } health_report { overall_health: HEALTHY } } }
I20260926 16:32:12.491917 12917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:12.488320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33396 (local address 127.11.8.195:33629)
0926 16:32:12.488437 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:12.488441 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:12.488456 (+ 15us) server_negotiation.cc:407] Connection header received
0926 16:32:12.488580 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:12.488583 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:12.488635 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:12.488720 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:32:12.488933 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:12.489731 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:12.490663 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:12.491041 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:12.491139 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:12.491424 (+ 285us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:32:12.491506 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:32:12.491588 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:12.491713 (+ 125us) server_negotiation.cc:299] Negotiation successful
0926 16:32:12.491769 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 16:32:12.491945 12959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:12.488235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:33629 (local address 127.0.0.1:33396)
0926 16:32:12.488360 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:12.488375 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:12.488472 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:12.488741 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:12.488743 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:12.488750 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 16:32:12.488836 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:12.488840 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:12.489835 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:12.489838 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:12.490556 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:12.490567 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:12.491160 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:12.491162 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:12.491228 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:12.491273 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 16:32:12.491592 (+ 319us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 16:32:12.491610 (+ 18us) client_negotiation.cc:769] Sending connection context
0926 16:32:12.491690 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 16:32:12.491789 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260926 16:32:12.494529 12572 raft_consensus.cc:1261] T 43cb3454ab7848c18a217d0832ed9c64 P af9cfd84a2ad48f6864a099828976e6a [term 1 FOLLOWER]: Refusing update from remote peer 4ff1455b40274737b068e75621fc0fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:32:12.494537 12712 raft_consensus.cc:1261] T 43cb3454ab7848c18a217d0832ed9c64 P c84b627d26e04df5b0550e4f520f764a [term 1 FOLLOWER]: Refusing update from remote peer 4ff1455b40274737b068e75621fc0fb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:32:12.494948 12931 consensus_queue.cc:1035] T 43cb3454ab7848c18a217d0832ed9c64 P 4ff1455b40274737b068e75621fc0fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af9cfd84a2ad48f6864a099828976e6a" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:32:12.495083 12927 consensus_queue.cc:1035] T 43cb3454ab7848c18a217d0832ed9c64 P 4ff1455b40274737b068e75621fc0fb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:32:12.497224 12928 tablet_replica.cc:439] T 43cb3454ab7848c18a217d0832ed9c64 P af9cfd84a2ad48f6864a099828976e6a: TxnStatusTablet state changed. Reason: New leader 4ff1455b40274737b068e75621fc0fb9. Latest consensus state: current_term: 1 leader_uuid: "4ff1455b40274737b068e75621fc0fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af9cfd84a2ad48f6864a099828976e6a" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40449 } } peers { permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 } } peers { permanent_uuid: "4ff1455b40274737b068e75621fc0fb9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33629 } } }
I20260926 16:32:12.497349 12928 tablet_replica.cc:442] T 43cb3454ab7848c18a217d0832ed9c64 P af9cfd84a2ad48f6864a099828976e6a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 16:32:12.497677 12929 tablet_replica.cc:439] T 43cb3454ab7848c18a217d0832ed9c64 P c84b627d26e04df5b0550e4f520f764a: TxnStatusTablet state changed. Reason: New leader 4ff1455b40274737b068e75621fc0fb9. Latest consensus state: current_term: 1 leader_uuid: "4ff1455b40274737b068e75621fc0fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af9cfd84a2ad48f6864a099828976e6a" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40449 } } peers { permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 } } peers { permanent_uuid: "4ff1455b40274737b068e75621fc0fb9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33629 } } }
I20260926 16:32:12.497776 12929 tablet_replica.cc:442] T 43cb3454ab7848c18a217d0832ed9c64 P c84b627d26e04df5b0550e4f520f764a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 16:32:12.499037 12931 tablet_replica.cc:439] T 43cb3454ab7848c18a217d0832ed9c64 P 4ff1455b40274737b068e75621fc0fb9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4ff1455b40274737b068e75621fc0fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af9cfd84a2ad48f6864a099828976e6a" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40449 } } peers { permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 } } peers { permanent_uuid: "4ff1455b40274737b068e75621fc0fb9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33629 } } }
I20260926 16:32:12.499141 12931 tablet_replica.cc:442] T 43cb3454ab7848c18a217d0832ed9c64 P 4ff1455b40274737b068e75621fc0fb9: This TxnStatusTablet replica's current role is: LEADER
I20260926 16:32:12.501061 12927 tablet_replica.cc:439] T 43cb3454ab7848c18a217d0832ed9c64 P 4ff1455b40274737b068e75621fc0fb9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4ff1455b40274737b068e75621fc0fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af9cfd84a2ad48f6864a099828976e6a" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40449 } } peers { permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 } } peers { permanent_uuid: "4ff1455b40274737b068e75621fc0fb9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33629 } } }
I20260926 16:32:12.501173 12927 tablet_replica.cc:442] T 43cb3454ab7848c18a217d0832ed9c64 P 4ff1455b40274737b068e75621fc0fb9: This TxnStatusTablet replica's current role is: LEADER
I20260926 16:32:12.502501 12929 tablet_replica.cc:439] T 43cb3454ab7848c18a217d0832ed9c64 P c84b627d26e04df5b0550e4f520f764a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4ff1455b40274737b068e75621fc0fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af9cfd84a2ad48f6864a099828976e6a" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40449 } } peers { permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 } } peers { permanent_uuid: "4ff1455b40274737b068e75621fc0fb9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33629 } } }
I20260926 16:32:12.502604 12929 tablet_replica.cc:442] T 43cb3454ab7848c18a217d0832ed9c64 P c84b627d26e04df5b0550e4f520f764a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 16:32:12.502779 12928 tablet_replica.cc:439] T 43cb3454ab7848c18a217d0832ed9c64 P af9cfd84a2ad48f6864a099828976e6a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4ff1455b40274737b068e75621fc0fb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af9cfd84a2ad48f6864a099828976e6a" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40449 } } peers { permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 } } peers { permanent_uuid: "4ff1455b40274737b068e75621fc0fb9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33629 } } }
I20260926 16:32:12.502888 12928 tablet_replica.cc:442] T 43cb3454ab7848c18a217d0832ed9c64 P af9cfd84a2ad48f6864a099828976e6a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 16:32:12.509721 12969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:12.505250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50678 (local address 127.11.8.193:40449)
0926 16:32:12.505503 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:12.505515 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:12.505525 (+ 10us) server_negotiation.cc:407] Connection header received
0926 16:32:12.505583 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:12.505585 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:12.505631 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:12.505755 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:32:12.506021 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:12.506945 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:12.507883 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:12.508188 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:12.508265 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:12.508594 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:32:12.509139 (+ 545us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:32:12.509226 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:12.509338 (+ 112us) server_negotiation.cc:299] Negotiation successful
0926 16:32:12.509383 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":116,"threads_started":1}
I20260926 16:32:12.519865 12917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:12.516492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:38445 (local address 127.11.8.195:33629)
0926 16:32:12.516613 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:12.516619 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:12.516829 (+ 210us) server_negotiation.cc:407] Connection header received
0926 16:32:12.516925 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:12.516928 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:12.516977 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:12.517058 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:32:12.517334 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:12.517855 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:12.518856 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:12.519067 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:12.519172 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:12.519456 (+ 284us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:32:12.519517 (+ 61us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:32:12.519592 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:12.519680 (+ 88us) server_negotiation.cc:299] Negotiation successful
0926 16:32:12.519727 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 16:32:12.519881 12972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:12.516413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:33629 (local address 127.11.8.193:38445)
0926 16:32:12.516712 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:12.516724 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:12.516835 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:12.517068 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:12.517071 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:12.517078 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 16:32:12.517212 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:12.517218 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:12.517954 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:12.517961 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:12.518732 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:12.518750 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:12.519171 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:12.519174 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:12.519280 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:12.519319 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 16:32:12.519596 (+ 277us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 16:32:12.519605 (+ 9us) client_negotiation.cc:769] Sending connection context
0926 16:32:12.519670 (+ 65us) client_negotiation.cc:240] Negotiation successful
0926 16:32:12.519743 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":103,"threads_started":1}
I20260926 16:32:12.526769 12712 raft_consensus.cc:1261] T 6965e80d53af47ab99e1c5f348f1ed4a P c84b627d26e04df5b0550e4f520f764a [term 1 FOLLOWER]: Refusing update from remote peer af9cfd84a2ad48f6864a099828976e6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:32:12.526897 12852 raft_consensus.cc:1261] T 6965e80d53af47ab99e1c5f348f1ed4a P 4ff1455b40274737b068e75621fc0fb9 [term 1 FOLLOWER]: Refusing update from remote peer af9cfd84a2ad48f6864a099828976e6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:32:12.527222 12928 consensus_queue.cc:1035] T 6965e80d53af47ab99e1c5f348f1ed4a P af9cfd84a2ad48f6864a099828976e6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c84b627d26e04df5b0550e4f520f764a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 44717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:32:12.527464 12951 consensus_queue.cc:1035] T 6965e80d53af47ab99e1c5f348f1ed4a P af9cfd84a2ad48f6864a099828976e6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ff1455b40274737b068e75621fc0fb9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:32:12.531047 12936 mvcc.cc:205] Tried to move back new op lower bound from 7333643602025271296 to 7333643600919621632. Current Snapshot: MvccSnapshot[applied={T|T < 7333643602025271296}]
I20260926 16:32:12.532787 12934 mvcc.cc:205] Tried to move back new op lower bound from 7333643602025271296 to 7333643600919621632. Current Snapshot: MvccSnapshot[applied={T|T < 7333643602025271296}]
I20260926 16:32:12.552174 12969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:12.548328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:46305 (local address 127.11.8.193:40449)
0926 16:32:12.548457 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:12.548465 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:12.548738 (+ 273us) server_negotiation.cc:407] Connection header received
0926 16:32:12.548884 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:12.548887 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:12.548937 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:12.549022 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:32:12.549323 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:12.550020 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:12.551038 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:12.551348 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:12.551458 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:12.551737 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:32:12.551814 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:32:12.551898 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:12.551973 (+ 75us) server_negotiation.cc:299] Negotiation successful
0926 16:32:12.552023 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 16:32:12.552175 12977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:12.548251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:40449 (local address 127.11.8.195:46305)
0926 16:32:12.548623 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:12.548640 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:12.548749 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:12.549046 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:12.549049 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:12.549057 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 16:32:12.549173 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:12.549179 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:12.550125 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:12.550132 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:12.550910 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:12.550920 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:12.551454 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:12.551458 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:12.551550 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:12.551600 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 16:32:12.551894 (+ 294us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 16:32:12.551900 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 16:32:12.551965 (+ 65us) client_negotiation.cc:240] Negotiation successful
0926 16:32:12.552044 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":106,"threads_started":1}
I20260926 16:32:12.585688 12414 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47068:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:32:12.585873 12414 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47068:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:32:12.588327 12414 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5303c2fae4bb4b36befb691bd3c73723: Sending DeleteTablet for 3 replicas of tablet 6965e80d53af47ab99e1c5f348f1ed4a
I20260926 16:32:12.589113 12692 tablet_service.cc:1514] Processing DeleteTablet for tablet 6965e80d53af47ab99e1c5f348f1ed4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:32:12 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:42488
I20260926 16:32:12.589306 12943 tablet_replica.cc:330] T 6965e80d53af47ab99e1c5f348f1ed4a P c84b627d26e04df5b0550e4f520f764a: stopping tablet replica
I20260926 16:32:12.589447 12943 raft_consensus.cc:2229] T 6965e80d53af47ab99e1c5f348f1ed4a P c84b627d26e04df5b0550e4f520f764a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:32:12.589591 12943 raft_consensus.cc:2258] T 6965e80d53af47ab99e1c5f348f1ed4a P c84b627d26e04df5b0550e4f520f764a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:32:12.589821 12552 tablet_service.cc:1514] Processing DeleteTablet for tablet 6965e80d53af47ab99e1c5f348f1ed4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:32:12 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:50666
I20260926 16:32:12.590023 12945 tablet_replica.cc:330] T 6965e80d53af47ab99e1c5f348f1ed4a P af9cfd84a2ad48f6864a099828976e6a: stopping tablet replica
I20260926 16:32:12.590143 12945 raft_consensus.cc:2229] T 6965e80d53af47ab99e1c5f348f1ed4a P af9cfd84a2ad48f6864a099828976e6a [term 1 LEADER]: Raft consensus shutting down.
I20260926 16:32:12.590489 12945 raft_consensus.cc:2258] T 6965e80d53af47ab99e1c5f348f1ed4a P af9cfd84a2ad48f6864a099828976e6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:32:12.590509 12943 ts_tablet_manager.cc:1905] T 6965e80d53af47ab99e1c5f348f1ed4a P c84b627d26e04df5b0550e4f520f764a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:32:12.591231 12945 ts_tablet_manager.cc:1905] T 6965e80d53af47ab99e1c5f348f1ed4a P af9cfd84a2ad48f6864a099828976e6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:32:12.593528 12943 ts_tablet_manager.cc:1918] T 6965e80d53af47ab99e1c5f348f1ed4a P c84b627d26e04df5b0550e4f520f764a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 16:32:12.593611 12943 log.cc:1196] T 6965e80d53af47ab99e1c5f348f1ed4a P c84b627d26e04df5b0550e4f520f764a: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-1/wal/wals/6965e80d53af47ab99e1c5f348f1ed4a
I20260926 16:32:12.593880 12943 ts_tablet_manager.cc:1939] T 6965e80d53af47ab99e1c5f348f1ed4a P c84b627d26e04df5b0550e4f520f764a: Deleting consensus metadata
I20260926 16:32:12.594349 12945 ts_tablet_manager.cc:1918] T 6965e80d53af47ab99e1c5f348f1ed4a P af9cfd84a2ad48f6864a099828976e6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 16:32:12.594429 12945 log.cc:1196] T 6965e80d53af47ab99e1c5f348f1ed4a P af9cfd84a2ad48f6864a099828976e6a: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-0/wal/wals/6965e80d53af47ab99e1c5f348f1ed4a
I20260926 16:32:12.594703 12402 catalog_manager.cc:4841] TS c84b627d26e04df5b0550e4f520f764a (127.11.8.194:44717): tablet 6965e80d53af47ab99e1c5f348f1ed4a (table test-table [id=76cbfd4011754caea499838bc678b005]) successfully deleted
I20260926 16:32:12.595058 12832 tablet_service.cc:1514] Processing DeleteTablet for tablet 6965e80d53af47ab99e1c5f348f1ed4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:32:12 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:33372
I20260926 16:32:12.595232 12944 tablet_replica.cc:330] T 6965e80d53af47ab99e1c5f348f1ed4a P 4ff1455b40274737b068e75621fc0fb9: stopping tablet replica
I20260926 16:32:12.595364 12944 raft_consensus.cc:2229] T 6965e80d53af47ab99e1c5f348f1ed4a P 4ff1455b40274737b068e75621fc0fb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:32:12.595482 12944 raft_consensus.cc:2258] T 6965e80d53af47ab99e1c5f348f1ed4a P 4ff1455b40274737b068e75621fc0fb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:32:12.597266 12944 ts_tablet_manager.cc:1905] T 6965e80d53af47ab99e1c5f348f1ed4a P 4ff1455b40274737b068e75621fc0fb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:32:12.600070 12944 ts_tablet_manager.cc:1918] T 6965e80d53af47ab99e1c5f348f1ed4a P 4ff1455b40274737b068e75621fc0fb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 16:32:12.600150 12944 log.cc:1196] T 6965e80d53af47ab99e1c5f348f1ed4a P 4ff1455b40274737b068e75621fc0fb9: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790440324823152-11299-0/minicluster-data/ts-2/wal/wals/6965e80d53af47ab99e1c5f348f1ed4a
I20260926 16:32:12.600414 12944 ts_tablet_manager.cc:1939] T 6965e80d53af47ab99e1c5f348f1ed4a P 4ff1455b40274737b068e75621fc0fb9: Deleting consensus metadata
I20260926 16:32:12.601171 12401 catalog_manager.cc:4841] TS 4ff1455b40274737b068e75621fc0fb9 (127.11.8.195:33629): tablet 6965e80d53af47ab99e1c5f348f1ed4a (table test-table [id=76cbfd4011754caea499838bc678b005]) successfully deleted
I20260926 16:32:12.601588 12945 ts_tablet_manager.cc:1939] T 6965e80d53af47ab99e1c5f348f1ed4a P af9cfd84a2ad48f6864a099828976e6a: Deleting consensus metadata
I20260926 16:32:12.602325 12399 catalog_manager.cc:4841] TS af9cfd84a2ad48f6864a099828976e6a (127.11.8.193:40449): tablet 6965e80d53af47ab99e1c5f348f1ed4a (table test-table [id=76cbfd4011754caea499838bc678b005]) successfully deleted
Sep 26 16:32:12 dist-test-slave-2zvq krb5kdc[12370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440331, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.8.193@KRBTEST.COM
I20260926 16:32:12.616382 12969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:12.602879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50684 (local address 127.11.8.193:40449)
0926 16:32:12.602992 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:12.602996 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:12.603570 (+ 574us) server_negotiation.cc:407] Connection header received
0926 16:32:12.603748 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:12.603751 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:12.603798 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:12.603849 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:12.604866 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:12.605665 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:12.606916 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:12.607226 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:12.607322 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:12.611589 (+ 4267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:12.611604 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:12.611625 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:12.611660 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:12.613944 (+ 2284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:12.614592 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:12.614599 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:12.614601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:12.614719 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:12.615057 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:12.615061 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:12.615062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:12.615418 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:12.615623 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:12.616023 (+ 400us) server_negotiation.cc:299] Negotiation successful
0926 16:32:12.616167 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260926 16:32:12.617707 12612 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50684
I20260926 16:32:12.629076 12958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:12.619942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47088 (local address 127.11.8.254:36907)
0926 16:32:12.620097 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:12.620102 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:12.620204 (+ 102us) server_negotiation.cc:407] Connection header received
0926 16:32:12.620329 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:12.620333 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:12.620386 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:12.620477 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:12.621247 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:12.622087 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:12.623243 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:12.623592 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:12.623713 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:12.625273 (+ 1560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:12.625277 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:12.625279 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:12.625315 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:12.627030 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:12.627686 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:12.627690 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:12.627692 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:12.627788 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:12.628140 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:12.628144 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:12.628146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:12.628371 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:12.628540 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:12.628784 (+ 244us) server_negotiation.cc:299] Negotiation successful
0926 16:32:12.628913 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260926 16:32:12.629915 12414 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47088
I20260926 16:32:12.632035 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 12487
I20260926 16:32:12.639652 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 12627
I20260926 16:32:12.646924 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 12767
I20260926 16:32:12.653633 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 12383
2026-09-26T16:32:12Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3090 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:32:12 dist-test-slave-2zvq krb5kdc[12987](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:32:12 dist-test-slave-2zvq krb5kdc[12987](info): set up 2 sockets
Sep 26 16:32:12 dist-test-slave-2zvq krb5kdc[12987](info): commencing operation
krb5kdc: starting...
W20260926 16:32:13.683755 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:32:13 dist-test-slave-2zvq krb5kdc[12987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:32:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:32:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
I20260926 16:32:13.801054 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:34553
--webserver_interface=127.11.8.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:41679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:34553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:13.910039 13000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:13.910375 13000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:13.910422 13000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:13.914327 13000 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:32:13.914377 13000 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 16:32:13.914394 13000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:13.914412 13000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:13.914428 13000 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:32:13.914443 13000 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:32:13.919703 13000 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:41679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:34553
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:34553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13000
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:13.920677 13000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:13.921631 13000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:13.929904 13008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:13.930060 13000 server_base.cc:1034] running on GCE node
W20260926 16:32:13.929870 13005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:13.929888 13006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:13.930780 13000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:13.931871 13000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:13.933063 13000 hybrid_clock.cc:648] HybridClock initialized: now 1790440333933044 us; error 34 us; skew 500 ppm
Sep 26 16:32:13 dist-test-slave-2zvq krb5kdc[12987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440333, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:13.936056 13000 init.cc:377] Logged in from keytab as kudu/127.11.8.254@KRBTEST.COM (short username kudu)
I20260926 16:32:13.937001 13000 webserver.cc:460] Webserver started at http://127.11.8.254:45029/ using document root <none> and password file <none>
I20260926 16:32:13.937564 13000 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:13.937652 13000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:13.937860 13000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:13.939937 13000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "3d59182d471a4c5596b65f20feab477e"
format_stamp: "Formatted at 2026-09-26 16:32:13 on dist-test-slave-2zvq"
server_key: "03ad838b45ca553c5b4f4c0ec3e3bcd3"
server_key_iv: "7109ed9fe7c3e361c75b03f0425a66d6"
server_key_version: "encryptionkey@0"
I20260926 16:32:13.940537 13000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "3d59182d471a4c5596b65f20feab477e"
format_stamp: "Formatted at 2026-09-26 16:32:13 on dist-test-slave-2zvq"
server_key: "03ad838b45ca553c5b4f4c0ec3e3bcd3"
server_key_iv: "7109ed9fe7c3e361c75b03f0425a66d6"
server_key_version: "encryptionkey@0"
I20260926 16:32:13.944356 13000 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260926 16:32:13.946803 13015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:13.948052 13000 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:32:13.948190 13000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "3d59182d471a4c5596b65f20feab477e"
format_stamp: "Formatted at 2026-09-26 16:32:13 on dist-test-slave-2zvq"
server_key: "03ad838b45ca553c5b4f4c0ec3e3bcd3"
server_key_iv: "7109ed9fe7c3e361c75b03f0425a66d6"
server_key_version: "encryptionkey@0"
I20260926 16:32:13.948321 13000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:13.985356 13000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:13.988754 13000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:13.988957 13000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:13.996636 13000 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:34553
I20260926 16:32:13.996659 13077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:34553 every 8 connection(s)
I20260926 16:32:13.997546 13000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:32:14.000275 13078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:14.005730 13078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d59182d471a4c5596b65f20feab477e: Bootstrap starting.
I20260926 16:32:14.007417 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 13000
I20260926 16:32:14.007531 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:32:14.007812 11299 external_mini_cluster.cc:1428] Setting key 2987a9a16fe07f1671656624e9c996f9
I20260926 16:32:14.008569 13078 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d59182d471a4c5596b65f20feab477e: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:14.009496 13078 log.cc:824] T 00000000000000000000000000000000 P 3d59182d471a4c5596b65f20feab477e: Log is configured to *not* fsync() on all Append() calls
I20260926 16:32:14.011504 13078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d59182d471a4c5596b65f20feab477e: No bootstrap required, opened a new log
I20260926 16:32:14.015074 13078 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d59182d471a4c5596b65f20feab477e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d59182d471a4c5596b65f20feab477e" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 34553 } }
I20260926 16:32:14.015347 13078 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d59182d471a4c5596b65f20feab477e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:14.015439 13078 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d59182d471a4c5596b65f20feab477e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d59182d471a4c5596b65f20feab477e, State: Initialized, Role: FOLLOWER
I20260926 16:32:14.015895 13078 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d59182d471a4c5596b65f20feab477e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d59182d471a4c5596b65f20feab477e" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 34553 } }
I20260926 16:32:14.016040 13078 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d59182d471a4c5596b65f20feab477e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:32:14.016124 13078 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d59182d471a4c5596b65f20feab477e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:32:14.016242 13078 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d59182d471a4c5596b65f20feab477e [term 0 FOLLOWER]: Advancing to term 1
Sep 26 16:32:14 dist-test-slave-2zvq krb5kdc[12987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:32:14.017328 13078 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d59182d471a4c5596b65f20feab477e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d59182d471a4c5596b65f20feab477e" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 34553 } }
I20260926 16:32:14.017668 13078 leader_election.cc:302] T 00000000000000000000000000000000 P 3d59182d471a4c5596b65f20feab477e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d59182d471a4c5596b65f20feab477e; no voters:
I20260926 16:32:14.018086 13078 leader_election.cc:288] T 00000000000000000000000000000000 P 3d59182d471a4c5596b65f20feab477e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:32:14.018168 13083 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d59182d471a4c5596b65f20feab477e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:32:14.018450 13083 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d59182d471a4c5596b65f20feab477e [term 1 LEADER]: Becoming Leader. State: Replica: 3d59182d471a4c5596b65f20feab477e, State: Running, Role: LEADER
I20260926 16:32:14.018821 13083 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d59182d471a4c5596b65f20feab477e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d59182d471a4c5596b65f20feab477e" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 34553 } }
I20260926 16:32:14.019610 13078 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d59182d471a4c5596b65f20feab477e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:32:14.019953 13085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d59182d471a4c5596b65f20feab477e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d59182d471a4c5596b65f20feab477e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d59182d471a4c5596b65f20feab477e" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 34553 } } }
I20260926 16:32:14.019954 13083 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d59182d471a4c5596b65f20feab477e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d59182d471a4c5596b65f20feab477e. Latest consensus state: current_term: 1 leader_uuid: "3d59182d471a4c5596b65f20feab477e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d59182d471a4c5596b65f20feab477e" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 34553 } } }
I20260926 16:32:14.020184 13085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d59182d471a4c5596b65f20feab477e [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:14.020292 13083 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d59182d471a4c5596b65f20feab477e [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:14.020664 13088 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:32:14.023137 13081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:14.008930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45794 (local address 127.11.8.254:34553)
0926 16:32:14.009364 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:14.009378 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:14.009415 (+ 37us) server_negotiation.cc:407] Connection header received
0926 16:32:14.010356 (+ 941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:14.010377 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:14.010705 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:14.011060 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:14.011910 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.012891 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.013791 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.014380 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.014561 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:14.017294 (+ 2733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:14.017300 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:14.017313 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:14.017341 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:14.019533 (+ 2192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:14.020089 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:14.020097 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:14.020103 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:14.020276 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:14.020625 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:14.020629 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:14.020631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:14.021129 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:14.021303 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:14.021773 (+ 470us) server_negotiation.cc:299] Negotiation successful
0926 16:32:14.022086 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":133,"threads_started":1}
I20260926 16:32:14.024284 13088 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:32:14.030018 13088 catalog_manager.cc:1347] Generated new cluster ID: a9bb134c91bf45d8bf8d2aa83f9d02f3
I20260926 16:32:14.030092 13088 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:32:14.041988 13088 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:32:14.042968 13088 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:32:14.054116 13088 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d59182d471a4c5596b65f20feab477e: Generated new TSK 0
I20260926 16:32:14.055078 13088 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.8.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab.
Entry for principal kudu/127.11.8.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab.
I20260926 16:32:14.090113 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.193:0
--local_ip_for_outbound_sockets=127.11.8.193
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:34553
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:41679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:14.204118 13104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:14.204355 13104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:14.204398 13104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:14.208328 13104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:14.208377 13104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:14.208415 13104 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 16:32:14.208429 13104 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 16:32:14.208459 13104 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.193
I20260926 16:32:14.213408 13104 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:41679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.8.254:34553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.193
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:14.214435 13104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:14.215408 13104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:14.225111 13110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:14.225111 13112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:14.225200 13109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:14.225376 13104 server_base.cc:1034] running on GCE node
I20260926 16:32:14.225853 13104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:14.226577 13104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:14.227792 13104 hybrid_clock.cc:648] HybridClock initialized: now 1790440334227780 us; error 33 us; skew 500 ppm
Sep 26 16:32:14 dist-test-slave-2zvq krb5kdc[12987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440334, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:14.230562 13104 init.cc:377] Logged in from keytab as kudu/127.11.8.193@KRBTEST.COM (short username kudu)
I20260926 16:32:14.231321 13104 webserver.cc:460] Webserver started at http://127.11.8.193:40745/ using document root <none> and password file <none>
I20260926 16:32:14.231819 13104 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:14.231880 13104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:14.232038 13104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:14.234020 13104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-0/data/instance:
uuid: "8c2dc32c75904237a8d40ae792d3c57e"
format_stamp: "Formatted at 2026-09-26 16:32:14 on dist-test-slave-2zvq"
server_key: "e4ba767272b7ab8b362c046b9351d045"
server_key_iv: "3bae14b899801a2452cfae5c9736af95"
server_key_version: "encryptionkey@0"
I20260926 16:32:14.234625 13104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance:
uuid: "8c2dc32c75904237a8d40ae792d3c57e"
format_stamp: "Formatted at 2026-09-26 16:32:14 on dist-test-slave-2zvq"
server_key: "e4ba767272b7ab8b362c046b9351d045"
server_key_iv: "3bae14b899801a2452cfae5c9736af95"
server_key_version: "encryptionkey@0"
I20260926 16:32:14.238214 13104 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260926 16:32:14.240545 13119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:14.241757 13104 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260926 16:32:14.241906 13104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-0/wal
uuid: "8c2dc32c75904237a8d40ae792d3c57e"
format_stamp: "Formatted at 2026-09-26 16:32:14 on dist-test-slave-2zvq"
server_key: "e4ba767272b7ab8b362c046b9351d045"
server_key_iv: "3bae14b899801a2452cfae5c9736af95"
server_key_version: "encryptionkey@0"
I20260926 16:32:14.242005 13104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:14.256992 13104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:14.260438 13104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:14.260641 13104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:14.262055 13104 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:14.262101 13104 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:14.262140 13104 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:14.262193 13104 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 16:32:14 dist-test-slave-2zvq krb5kdc[12987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440334, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:32:14.273605 13104 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.193:33355
I20260926 16:32:14.273674 13234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.193:33355 every 8 connection(s)
I20260926 16:32:14.274571 13104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
I20260926 16:32:14.276139 13081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:14.262997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:42169 (local address 127.11.8.254:34553)
0926 16:32:14.263144 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:14.263149 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:14.263950 (+ 801us) server_negotiation.cc:407] Connection header received
0926 16:32:14.265118 (+ 1168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:14.265122 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:14.265205 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:14.265262 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:14.267057 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.267740 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.268969 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.269281 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.269400 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:14.272364 (+ 2964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:14.272368 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:14.272371 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:14.272403 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:14.273957 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:14.274524 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:14.274530 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:14.274532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:14.274641 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:14.275026 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:14.275029 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:14.275031 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:14.275237 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:14.275335 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:14.275916 (+ 581us) server_negotiation.cc:299] Negotiation successful
0926 16:32:14.276023 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 16:32:14.276924 13128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:14.263361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:34553 (local address 127.11.8.193:42169)
0926 16:32:14.263802 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:14.263852 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:14.264833 (+ 981us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:14.265757 (+ 924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:14.265766 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:14.266199 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:14.266814 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:14.266850 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.267871 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.267875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:14.268842 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:14.268851 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.269467 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.269472 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:14.269640 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:14.270232 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:14.270257 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:14.272197 (+ 1940us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:14.274086 (+ 1889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:14.274092 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:14.274108 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:14.274422 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:14.274731 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:14.274734 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:14.274736 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:14.274893 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:14.275385 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:14.275397 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:14.275668 (+ 271us) client_negotiation.cc:769] Sending connection context
0926 16:32:14.275815 (+ 147us) client_negotiation.cc:240] Negotiation successful
0926 16:32:14.276055 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":123,"threads_started":1}
I20260926 16:32:14.277247 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 13104
I20260926 16:32:14.277357 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance
I20260926 16:32:14.277704 11299 external_mini_cluster.cc:1428] Setting key ce905c58589d81a11c062e41b97bfa6f
I20260926 16:32:14.278538 13235 heartbeater.cc:347] Connected to a master server at 127.11.8.254:34553
I20260926 16:32:14.278677 13235 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:14.279214 13235 heartbeater.cc:511] Master 127.11.8.254:34553 requested a full tablet report, sending...
I20260926 16:32:14.280665 13032 ts_manager.cc:194] Registered new tserver with Master: 8c2dc32c75904237a8d40ae792d3c57e (127.11.8.193:33355)
I20260926 16:32:14.281785 13032 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.193@KRBTEST.COM'} at 127.11.8.193:42169
WARNING: no policy specified for kudu/127.11.8.194@KRBTEST.COM; defaulting to no policy
I20260926 16:32:14.289573 13242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:14.280904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:34553 (local address 127.11.8.193:33881)
0926 16:32:14.282447 (+ 1543us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:14.282463 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:14.282563 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:14.282834 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:14.282838 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:14.283074 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:14.283186 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:14.283193 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.284299 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.284301 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:14.284837 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:14.284844 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.285527 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.285529 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:14.285601 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:14.286092 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:14.286103 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:14.286452 (+ 349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:14.288241 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:14.288243 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:14.288245 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:14.288431 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:14.288695 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:14.288698 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:14.288699 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:14.288783 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:14.289174 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:14.289176 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:14.289250 (+ 74us) client_negotiation.cc:769] Sending connection context
0926 16:32:14.289333 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 16:32:14.289441 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1464,"thread_start_us":92,"threads_started":1}
I20260926 16:32:14.289578 13081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:14.280712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:33881 (local address 127.11.8.254:34553)
0926 16:32:14.280860 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:14.280874 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:14.282546 (+ 1672us) server_negotiation.cc:407] Connection header received
0926 16:32:14.282658 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:14.282663 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:14.282717 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:14.282798 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:14.283312 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.284168 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.284985 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.285403 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.285521 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:14.286579 (+ 1058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:14.286583 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:14.286586 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:14.286613 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:14.288140 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:14.288516 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:14.288518 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:14.288519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:14.288588 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:14.288875 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:14.288878 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:14.288879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:14.289102 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:14.289177 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:14.289387 (+ 210us) server_negotiation.cc:299] Negotiation successful
0926 16:32:14.289476 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab.
Entry for principal kudu/127.11.8.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab.
I20260926 16:32:14.305436 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.194:0
--local_ip_for_outbound_sockets=127.11.8.194
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:34553
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:41679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:14.414003 13244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:14.414419 13244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:14.414516 13244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:14.418697 13244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:14.418779 13244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:14.418821 13244 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 16:32:14.418865 13244 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 16:32:14.418928 13244 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.194
I20260926 16:32:14.424674 13244 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:41679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.8.254:34553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13244
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.194
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:14.425747 13244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:14.426797 13244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:14.436585 13249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:14.436615 13250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:14.436764 13244 server_base.cc:1034] running on GCE node
W20260926 16:32:14.436576 13252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:14.437331 13244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:14.437891 13244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:14.439049 13244 hybrid_clock.cc:648] HybridClock initialized: now 1790440334439032 us; error 28 us; skew 500 ppm
Sep 26 16:32:14 dist-test-slave-2zvq krb5kdc[12987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440334, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:14.441859 13244 init.cc:377] Logged in from keytab as kudu/127.11.8.194@KRBTEST.COM (short username kudu)
I20260926 16:32:14.442880 13244 webserver.cc:460] Webserver started at http://127.11.8.194:35307/ using document root <none> and password file <none>
I20260926 16:32:14.443382 13244 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:14.443444 13244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:14.443600 13244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:14.445729 13244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-1/data/instance:
uuid: "ebf7b662b0724a89b0f660920b5f4e8e"
format_stamp: "Formatted at 2026-09-26 16:32:14 on dist-test-slave-2zvq"
server_key: "92dd42164b432a03a7849617992dd529"
server_key_iv: "0680a085084366df15c333ce4090911c"
server_key_version: "encryptionkey@0"
I20260926 16:32:14.446506 13244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance:
uuid: "ebf7b662b0724a89b0f660920b5f4e8e"
format_stamp: "Formatted at 2026-09-26 16:32:14 on dist-test-slave-2zvq"
server_key: "92dd42164b432a03a7849617992dd529"
server_key_iv: "0680a085084366df15c333ce4090911c"
server_key_version: "encryptionkey@0"
I20260926 16:32:14.450212 13244 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:32:14.452697 13259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:14.454138 13244 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260926 16:32:14.454279 13244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-1/wal
uuid: "ebf7b662b0724a89b0f660920b5f4e8e"
format_stamp: "Formatted at 2026-09-26 16:32:14 on dist-test-slave-2zvq"
server_key: "92dd42164b432a03a7849617992dd529"
server_key_iv: "0680a085084366df15c333ce4090911c"
server_key_version: "encryptionkey@0"
I20260926 16:32:14.454437 13244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:14.486405 13244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:14.489327 13244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:14.489528 13244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:14.491159 13244 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:14.491225 13244 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:14.491276 13244 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:14.491307 13244 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 16:32:14 dist-test-slave-2zvq krb5kdc[12987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440334, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:32:14.502097 13244 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.194:43943
I20260926 16:32:14.502125 13374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.194:43943 every 8 connection(s)
I20260926 16:32:14.503012 13244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
I20260926 16:32:14.505441 13081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:14.491866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:34825 (local address 127.11.8.254:34553)
0926 16:32:14.492029 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:14.492036 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:14.493454 (+ 1418us) server_negotiation.cc:407] Connection header received
0926 16:32:14.494993 (+ 1539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:14.494996 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:14.495076 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:14.495179 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:14.496388 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.497278 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.498557 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.498840 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.498942 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:14.501609 (+ 2667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:14.501615 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:14.501620 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:14.501650 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:14.503381 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:14.503981 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:14.503985 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:14.503987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:14.504070 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:14.504410 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:14.504413 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:14.504415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:14.504595 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:14.504700 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:14.505157 (+ 457us) server_negotiation.cc:299] Negotiation successful
0926 16:32:14.505274 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 16:32:14.506187 13268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:14.492755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:34553 (local address 127.11.8.194:34825)
0926 16:32:14.493263 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:14.493326 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:14.494739 (+ 1413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:14.495365 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:14.495372 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:14.495747 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:14.496237 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:14.496250 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.497390 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.497393 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:14.498439 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:14.498449 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.498983 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.498987 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:14.499150 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:14.499795 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:14.499824 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:14.501371 (+ 1547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:14.503493 (+ 2122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:14.503498 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:14.503513 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:14.503878 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:14.504164 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:14.504168 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:14.504170 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:14.504308 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:14.504696 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:14.504702 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:14.504937 (+ 235us) client_negotiation.cc:769] Sending connection context
0926 16:32:14.505116 (+ 179us) client_negotiation.cc:240] Negotiation successful
0926 16:32:14.505426 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":106,"threads_started":1}
I20260926 16:32:14.507354 13375 heartbeater.cc:347] Connected to a master server at 127.11.8.254:34553
I20260926 16:32:14.507525 13375 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:14.508052 13375 heartbeater.cc:511] Master 127.11.8.254:34553 requested a full tablet report, sending...
I20260926 16:32:14.509063 13031 ts_manager.cc:194] Registered new tserver with Master: ebf7b662b0724a89b0f660920b5f4e8e (127.11.8.194:43943)
I20260926 16:32:14.509781 13031 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.194@KRBTEST.COM'} at 127.11.8.194:34825
I20260926 16:32:14.511838 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 13244
I20260926 16:32:14.511983 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance
I20260926 16:32:14.512249 11299 external_mini_cluster.cc:1428] Setting key b8f7683c616900298daebc3db307ff03
I20260926 16:32:14.516376 13381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:14.509330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:34553 (local address 127.11.8.194:55411)
0926 16:32:14.509586 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:14.509598 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:14.509696 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:14.509942 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:14.509945 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:14.510134 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:14.510227 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:14.510233 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.511269 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.511272 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:14.511817 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:14.511825 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.512310 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.512313 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:14.512412 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:14.512913 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:14.512925 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:14.513229 (+ 304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:14.514905 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:14.514908 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:14.514910 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:14.515103 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:14.515422 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:14.515425 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:14.515426 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:14.515505 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:14.515991 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:14.515993 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:14.516063 (+ 70us) client_negotiation.cc:769] Sending connection context
0926 16:32:14.516142 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 16:32:14.516243 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":103,"threads_started":1}
I20260926 16:32:14.516533 13081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:14.509586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:55411 (local address 127.11.8.254:34553)
0926 16:32:14.509704 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:14.509708 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:14.509722 (+ 14us) server_negotiation.cc:407] Connection header received
0926 16:32:14.509764 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:14.509768 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:14.509810 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:14.509935 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:14.510413 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.511139 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.511945 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.512202 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.512309 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:14.513365 (+ 1056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:14.513368 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:14.513371 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:14.513394 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:14.514788 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:14.515201 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:14.515207 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:14.515211 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:14.515304 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:14.515598 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:14.515602 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:14.515605 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:14.515881 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:14.516000 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:14.516226 (+ 226us) server_negotiation.cc:299] Negotiation successful
0926 16:32:14.516354 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
WARNING: no policy specified for kudu/127.11.8.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab.
Entry for principal kudu/127.11.8.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab.
I20260926 16:32:14.537472 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.195:0
--local_ip_for_outbound_sockets=127.11.8.195
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:34553
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:41679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:14.649097 13384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:14.649345 13384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:14.649389 13384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:14.653510 13384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:14.653578 13384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:14.653620 13384 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 16:32:14.653633 13384 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 16:32:14.653666 13384 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.195
I20260926 16:32:14.659116 13384 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:41679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.8.254:34553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.195
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:14.660130 13384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:14.661108 13384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:14.670534 13392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:14.670562 13389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:14.670562 13390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:14.670614 13384 server_base.cc:1034] running on GCE node
I20260926 16:32:14.671134 13384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:14.671692 13384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:14.672874 13384 hybrid_clock.cc:648] HybridClock initialized: now 1790440334672863 us; error 28 us; skew 500 ppm
Sep 26 16:32:14 dist-test-slave-2zvq krb5kdc[12987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440334, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:14.675577 13384 init.cc:377] Logged in from keytab as kudu/127.11.8.195@KRBTEST.COM (short username kudu)
I20260926 16:32:14.676445 13384 webserver.cc:460] Webserver started at http://127.11.8.195:46363/ using document root <none> and password file <none>
I20260926 16:32:14.677040 13384 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:14.677129 13384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:14.677346 13384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:14.679626 13384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-2/data/instance:
uuid: "0003ec5f5cdd45f291cc2505083bcd8e"
format_stamp: "Formatted at 2026-09-26 16:32:14 on dist-test-slave-2zvq"
server_key: "8b1f7115f67338b2dbe5b222a073c118"
server_key_iv: "0e988e485a9f2ca708655406837312c2"
server_key_version: "encryptionkey@0"
I20260926 16:32:14.680183 13384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance:
uuid: "0003ec5f5cdd45f291cc2505083bcd8e"
format_stamp: "Formatted at 2026-09-26 16:32:14 on dist-test-slave-2zvq"
server_key: "8b1f7115f67338b2dbe5b222a073c118"
server_key_iv: "0e988e485a9f2ca708655406837312c2"
server_key_version: "encryptionkey@0"
I20260926 16:32:14.683877 13384 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 16:32:14.686187 13399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:14.687439 13384 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:32:14.687574 13384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-2/wal
uuid: "0003ec5f5cdd45f291cc2505083bcd8e"
format_stamp: "Formatted at 2026-09-26 16:32:14 on dist-test-slave-2zvq"
server_key: "8b1f7115f67338b2dbe5b222a073c118"
server_key_iv: "0e988e485a9f2ca708655406837312c2"
server_key_version: "encryptionkey@0"
I20260926 16:32:14.687685 13384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:14.709465 13384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:14.712821 13384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:14.713027 13384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:14.714489 13384 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:14.714560 13384 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:14.714633 13384 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:14.714676 13384 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 16:32:14 dist-test-slave-2zvq krb5kdc[12987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440334, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:32:14.725708 13384 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.195:37479
I20260926 16:32:14.725783 13514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.195:37479 every 8 connection(s)
I20260926 16:32:14.726802 13384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
I20260926 16:32:14.728519 13081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:14.715390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:60131 (local address 127.11.8.254:34553)
0926 16:32:14.715568 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:14.715575 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:14.716435 (+ 860us) server_negotiation.cc:407] Connection header received
0926 16:32:14.717785 (+ 1350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:14.717790 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:14.717856 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:14.717922 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:14.719863 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.720542 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.721839 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.722169 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.722301 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:14.724729 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:14.724733 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:14.724735 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:14.724761 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:14.726371 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:14.726841 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:14.726845 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:14.726847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:14.726936 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:14.727348 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:14.727351 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:14.727352 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:14.727601 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:14.727715 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:14.728241 (+ 526us) server_negotiation.cc:299] Negotiation successful
0926 16:32:14.728356 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 16:32:14.729220 13408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:14.715778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:34553 (local address 127.11.8.195:60131)
0926 16:32:14.716256 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:14.716315 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:14.717488 (+ 1173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:14.718400 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:14.718410 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:14.718917 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:14.719680 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:14.719705 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.720667 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.720670 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:14.721707 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:14.721717 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.722358 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.722363 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:14.722568 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:14.723082 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:14.723097 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:14.724580 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:14.726478 (+ 1898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:14.726482 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:14.726493 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:14.726738 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:14.727032 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:14.727034 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:14.727035 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:14.727225 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:14.727720 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:14.727728 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:14.728008 (+ 280us) client_negotiation.cc:769] Sending connection context
0926 16:32:14.728188 (+ 180us) client_negotiation.cc:240] Negotiation successful
0926 16:32:14.728487 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":100,"threads_started":1}
I20260926 16:32:14.730454 13515 heartbeater.cc:347] Connected to a master server at 127.11.8.254:34553
I20260926 16:32:14.730610 13515 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:14.731127 13515 heartbeater.cc:511] Master 127.11.8.254:34553 requested a full tablet report, sending...
I20260926 16:32:14.732344 13032 ts_manager.cc:194] Registered new tserver with Master: 0003ec5f5cdd45f291cc2505083bcd8e (127.11.8.195:37479)
I20260926 16:32:14.732993 13032 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.195@KRBTEST.COM'} at 127.11.8.195:60131
I20260926 16:32:14.733850 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 13384
I20260926 16:32:14.733974 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance
I20260926 16:32:14.734341 11299 external_mini_cluster.cc:1428] Setting key a1355b3fdc591298f1cf98088a59eb32
I20260926 16:32:14.736404 11299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 16:32:14.740957 13081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:14.732516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:57289 (local address 127.11.8.254:34553)
0926 16:32:14.732639 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:14.732644 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:14.733523 (+ 879us) server_negotiation.cc:407] Connection header received
0926 16:32:14.733592 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:14.733595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:14.733646 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:14.733758 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:14.734327 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.735131 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.736207 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.736523 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.736625 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:14.737705 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:14.737712 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:14.737718 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:14.737752 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:14.739315 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:14.739741 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:14.739746 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:14.739748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:14.739839 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:14.740138 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:14.740143 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:14.740145 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:14.740320 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:14.740422 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:14.740691 (+ 269us) server_negotiation.cc:299] Negotiation successful
0926 16:32:14.740810 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 16:32:14.740964 13521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:14.732391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:34553 (local address 127.11.8.195:57289)
0926 16:32:14.733414 (+ 1023us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:14.733432 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:14.733516 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:14.733774 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:14.733777 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:14.734039 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:14.734162 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:14.734168 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.735296 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.735311 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:14.736073 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:14.736083 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.736631 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.736634 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:14.736713 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:14.737240 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:14.737252 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:14.737573 (+ 321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:14.739430 (+ 1857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:14.739435 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:14.739438 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:14.739640 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:14.739939 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:14.739942 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:14.739963 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:14.740049 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:14.740422 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:14.740425 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:14.740523 (+ 98us) client_negotiation.cc:769] Sending connection context
0926 16:32:14.740627 (+ 104us) client_negotiation.cc:240] Negotiation successful
0926 16:32:14.740770 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":930,"thread_start_us":106,"threads_started":1}
Sep 26 16:32:14 dist-test-slave-2zvq krb5kdc[12987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 16:32:14 dist-test-slave-2zvq krb5kdc[12987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:32:14.761731 13081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:14.753941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45808 (local address 127.11.8.254:34553)
0926 16:32:14.754081 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:14.754084 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:14.754097 (+ 13us) server_negotiation.cc:407] Connection header received
0926 16:32:14.754130 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:14.754132 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:14.754167 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:14.754231 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:14.754984 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.755463 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.756256 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.756453 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.756525 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:14.758892 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:14.758895 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:14.758897 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:14.758919 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:14.760316 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:14.760686 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:14.760689 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:14.760690 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:14.760757 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:14.761029 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:14.761031 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:14.761033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:14.761181 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:14.761273 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:14.761536 (+ 263us) server_negotiation.cc:299] Negotiation successful
0926 16:32:14.761619 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 16:32:14.764976 13032 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45808:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:32:14.768050 13032 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:32:14.783362 13534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:14.777757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55670 (local address 127.11.8.194:43943)
0926 16:32:14.778052 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:14.778066 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:14.778090 (+ 24us) server_negotiation.cc:407] Connection header received
0926 16:32:14.778168 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:14.778174 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:14.778460 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:14.779015 (+ 555us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:14.779137 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.780353 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.782039 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.782675 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.782801 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:14.782968 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:14.783114 (+ 146us) server_negotiation.cc:299] Negotiation successful
0926 16:32:14.783207 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":126,"threads_started":1}
I20260926 16:32:14.783360 13532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:14.776956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:43943 (local address 127.0.0.1:55670)
0926 16:32:14.777418 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:14.777471 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:14.777593 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:14.778664 (+ 1071us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:14.778670 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:14.778692 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:14.778866 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:14.778874 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.780471 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.780474 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:14.781941 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:14.781951 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.782803 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.782807 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:14.782920 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:14.782970 (+ 50us) client_negotiation.cc:769] Sending connection context
0926 16:32:14.783084 (+ 114us) client_negotiation.cc:240] Negotiation successful
0926 16:32:14.783200 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"mutex_wait_us":131,"thread_start_us":97,"threads_started":1}
I20260926 16:32:14.783597 13536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:14.778237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:37479 (local address 127.0.0.1:49088)
0926 16:32:14.779079 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:14.779090 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:14.779191 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:14.779613 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:14.779616 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:14.779632 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:14.779717 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:14.779723 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.780697 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.780700 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:14.781817 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:14.781826 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.783063 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.783069 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:14.783158 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:14.783180 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 16:32:14.783242 (+ 62us) client_negotiation.cc:240] Negotiation successful
0926 16:32:14.783305 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":782,"mutex_wait_us":28,"thread_start_us":95,"threads_started":1}
I20260926 16:32:14.783648 13537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:14.778403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49088 (local address 127.11.8.195:37479)
0926 16:32:14.778838 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:14.778851 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:14.779174 (+ 323us) server_negotiation.cc:407] Connection header received
0926 16:32:14.779294 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:14.779299 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:14.779495 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:14.779637 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:14.779813 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.780586 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.781927 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.782759 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.782854 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:14.783005 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:14.783425 (+ 420us) server_negotiation.cc:299] Negotiation successful
0926 16:32:14.783512 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":103,"threads_started":1}
I20260926 16:32:14.784476 13533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:14.777159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:33355 (local address 127.0.0.1:54260)
0926 16:32:14.778171 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:14.778184 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:14.778312 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:14.778983 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:14.778985 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:14.779001 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:14.779090 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:14.779097 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.780502 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.780505 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:14.783017 (+ 2512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:14.783029 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.784091 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.784098 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:14.784188 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:14.784210 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 16:32:14.784295 (+ 85us) client_negotiation.cc:240] Negotiation successful
0926 16:32:14.784355 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":953,"mutex_wait_us":67,"thread_start_us":105,"threads_started":1}
I20260926 16:32:14.784549 13535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:14.777814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54260 (local address 127.11.8.193:33355)
0926 16:32:14.778094 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:14.778103 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:14.778408 (+ 305us) server_negotiation.cc:407] Connection header received
0926 16:32:14.778487 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:14.778492 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:14.778683 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:14.778835 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:14.779217 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.780343 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.783146 (+ 2803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.783960 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.784067 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:14.784183 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:14.784321 (+ 138us) server_negotiation.cc:299] Negotiation successful
0926 16:32:14.784405 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":99,"threads_started":1}
I20260926 16:32:14.786616 13309 tablet_service.cc:1467] Processing CreateTablet for tablet 60626cc751cd4e32a766895981a084aa (DEFAULT_TABLE table=test-table [id=b0dc740d03b74ee78d17d48459b1669d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:14.786618 13449 tablet_service.cc:1467] Processing CreateTablet for tablet 60626cc751cd4e32a766895981a084aa (DEFAULT_TABLE table=test-table [id=b0dc740d03b74ee78d17d48459b1669d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:14.787058 13169 tablet_service.cc:1467] Processing CreateTablet for tablet 60626cc751cd4e32a766895981a084aa (DEFAULT_TABLE table=test-table [id=b0dc740d03b74ee78d17d48459b1669d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:14.787601 13309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60626cc751cd4e32a766895981a084aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:14.787616 13449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60626cc751cd4e32a766895981a084aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:14.787885 13169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60626cc751cd4e32a766895981a084aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:14.794162 13538 tablet_bootstrap.cc:492] T 60626cc751cd4e32a766895981a084aa P ebf7b662b0724a89b0f660920b5f4e8e: Bootstrap starting.
I20260926 16:32:14.795379 13539 tablet_bootstrap.cc:492] T 60626cc751cd4e32a766895981a084aa P 8c2dc32c75904237a8d40ae792d3c57e: Bootstrap starting.
I20260926 16:32:14.796175 13540 tablet_bootstrap.cc:492] T 60626cc751cd4e32a766895981a084aa P 0003ec5f5cdd45f291cc2505083bcd8e: Bootstrap starting.
I20260926 16:32:14.796334 13538 tablet_bootstrap.cc:654] T 60626cc751cd4e32a766895981a084aa P ebf7b662b0724a89b0f660920b5f4e8e: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:14.797235 13538 log.cc:824] T 60626cc751cd4e32a766895981a084aa P ebf7b662b0724a89b0f660920b5f4e8e: Log is configured to *not* fsync() on all Append() calls
I20260926 16:32:14.797353 13539 tablet_bootstrap.cc:654] T 60626cc751cd4e32a766895981a084aa P 8c2dc32c75904237a8d40ae792d3c57e: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:14.798161 13539 log.cc:824] T 60626cc751cd4e32a766895981a084aa P 8c2dc32c75904237a8d40ae792d3c57e: Log is configured to *not* fsync() on all Append() calls
I20260926 16:32:14.798249 13540 tablet_bootstrap.cc:654] T 60626cc751cd4e32a766895981a084aa P 0003ec5f5cdd45f291cc2505083bcd8e: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:14.799078 13540 log.cc:824] T 60626cc751cd4e32a766895981a084aa P 0003ec5f5cdd45f291cc2505083bcd8e: Log is configured to *not* fsync() on all Append() calls
I20260926 16:32:14.799335 13538 tablet_bootstrap.cc:492] T 60626cc751cd4e32a766895981a084aa P ebf7b662b0724a89b0f660920b5f4e8e: No bootstrap required, opened a new log
I20260926 16:32:14.799539 13538 ts_tablet_manager.cc:1397] T 60626cc751cd4e32a766895981a084aa P ebf7b662b0724a89b0f660920b5f4e8e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 16:32:14.800242 13539 tablet_bootstrap.cc:492] T 60626cc751cd4e32a766895981a084aa P 8c2dc32c75904237a8d40ae792d3c57e: No bootstrap required, opened a new log
I20260926 16:32:14.800446 13539 ts_tablet_manager.cc:1397] T 60626cc751cd4e32a766895981a084aa P 8c2dc32c75904237a8d40ae792d3c57e: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260926 16:32:14.801007 13540 tablet_bootstrap.cc:492] T 60626cc751cd4e32a766895981a084aa P 0003ec5f5cdd45f291cc2505083bcd8e: No bootstrap required, opened a new log
I20260926 16:32:14.801178 13540 ts_tablet_manager.cc:1397] T 60626cc751cd4e32a766895981a084aa P 0003ec5f5cdd45f291cc2505083bcd8e: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260926 16:32:14.802639 13538 raft_consensus.cc:348] T 60626cc751cd4e32a766895981a084aa P ebf7b662b0724a89b0f660920b5f4e8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:14.802867 13538 raft_consensus.cc:374] T 60626cc751cd4e32a766895981a084aa P ebf7b662b0724a89b0f660920b5f4e8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:14.802994 13538 raft_consensus.cc:729] T 60626cc751cd4e32a766895981a084aa P ebf7b662b0724a89b0f660920b5f4e8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebf7b662b0724a89b0f660920b5f4e8e, State: Initialized, Role: FOLLOWER
I20260926 16:32:14.803447 13538 consensus_queue.cc:260] T 60626cc751cd4e32a766895981a084aa P ebf7b662b0724a89b0f660920b5f4e8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:14.804073 13375 heartbeater.cc:503] Master 127.11.8.254:34553 was elected leader, sending a full tablet report...
I20260926 16:32:14.804437 13538 ts_tablet_manager.cc:1428] T 60626cc751cd4e32a766895981a084aa P ebf7b662b0724a89b0f660920b5f4e8e: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260926 16:32:14.804617 13539 raft_consensus.cc:348] T 60626cc751cd4e32a766895981a084aa P 8c2dc32c75904237a8d40ae792d3c57e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:14.804883 13539 raft_consensus.cc:374] T 60626cc751cd4e32a766895981a084aa P 8c2dc32c75904237a8d40ae792d3c57e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:14.804951 13539 raft_consensus.cc:729] T 60626cc751cd4e32a766895981a084aa P 8c2dc32c75904237a8d40ae792d3c57e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c2dc32c75904237a8d40ae792d3c57e, State: Initialized, Role: FOLLOWER
I20260926 16:32:14.805348 13539 consensus_queue.cc:260] T 60626cc751cd4e32a766895981a084aa P 8c2dc32c75904237a8d40ae792d3c57e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:14.805630 13540 raft_consensus.cc:348] T 60626cc751cd4e32a766895981a084aa P 0003ec5f5cdd45f291cc2505083bcd8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:14.805912 13540 raft_consensus.cc:374] T 60626cc751cd4e32a766895981a084aa P 0003ec5f5cdd45f291cc2505083bcd8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:14.805976 13235 heartbeater.cc:503] Master 127.11.8.254:34553 was elected leader, sending a full tablet report...
I20260926 16:32:14.806015 13540 raft_consensus.cc:729] T 60626cc751cd4e32a766895981a084aa P 0003ec5f5cdd45f291cc2505083bcd8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0003ec5f5cdd45f291cc2505083bcd8e, State: Initialized, Role: FOLLOWER
I20260926 16:32:14.806416 13540 consensus_queue.cc:260] T 60626cc751cd4e32a766895981a084aa P 0003ec5f5cdd45f291cc2505083bcd8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:14.807228 13539 ts_tablet_manager.cc:1428] T 60626cc751cd4e32a766895981a084aa P 8c2dc32c75904237a8d40ae792d3c57e: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260926 16:32:14.807323 13540 ts_tablet_manager.cc:1428] T 60626cc751cd4e32a766895981a084aa P 0003ec5f5cdd45f291cc2505083bcd8e: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260926 16:32:14.807695 13515 heartbeater.cc:503] Master 127.11.8.254:34553 was elected leader, sending a full tablet report...
I20260926 16:32:14.962116 13544 raft_consensus.cc:482] T 60626cc751cd4e32a766895981a084aa P ebf7b662b0724a89b0f660920b5f4e8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:32:14.962446 13544 raft_consensus.cc:504] T 60626cc751cd4e32a766895981a084aa P ebf7b662b0724a89b0f660920b5f4e8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:14.963636 13544 leader_election.cc:288] T 60626cc751cd4e32a766895981a084aa P ebf7b662b0724a89b0f660920b5f4e8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c2dc32c75904237a8d40ae792d3c57e (127.11.8.193:33355), 0003ec5f5cdd45f291cc2505083bcd8e (127.11.8.195:37479)
I20260926 16:32:14.968233 13268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:14.963905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:37479 (local address 127.11.8.194:53359)
0926 16:32:14.964094 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:14.964110 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:14.964204 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:14.964492 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:14.964494 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:14.964507 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:14.964663 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:14.964671 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.965752 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.965756 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:14.967101 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:14.967111 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.967856 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.967859 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:14.967944 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:14.967962 (+ 18us) client_negotiation.cc:769] Sending connection context
0926 16:32:14.968021 (+ 59us) client_negotiation.cc:240] Negotiation successful
0926 16:32:14.968079 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":3}
I20260926 16:32:14.968266 13537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:14.964122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:53359 (local address 127.11.8.195:37479)
0926 16:32:14.964238 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:14.964241 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:14.964252 (+ 11us) server_negotiation.cc:407] Connection header received
0926 16:32:14.964300 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:14.964302 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:14.964387 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:14.964477 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:14.964808 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.965617 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.967221 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.967749 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.967863 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:14.967915 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:14.968056 (+ 141us) server_negotiation.cc:299] Negotiation successful
0926 16:32:14.968122 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 16:32:14.968693 13535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:14.964006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:54915 (local address 127.11.8.193:33355)
0926 16:32:14.964135 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:14.964140 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:14.964352 (+ 212us) server_negotiation.cc:407] Connection header received
0926 16:32:14.964412 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:14.964418 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:14.964479 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:14.964550 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:14.964906 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.965648 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.967489 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.968106 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.968239 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:14.968300 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:14.968500 (+ 200us) server_negotiation.cc:299] Negotiation successful
0926 16:32:14.968574 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 16:32:14.969070 13547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:14.963936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:33355 (local address 127.11.8.194:54915)
0926 16:32:14.964245 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:14.964259 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:14.964342 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:14.964570 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:14.964572 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:14.964584 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:14.964739 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:14.964748 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.965752 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.965756 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:14.967375 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:14.967381 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:14.968229 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:14.968233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:14.968331 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:14.968359 (+ 28us) client_negotiation.cc:769] Sending connection context
0926 16:32:14.968830 (+ 471us) client_negotiation.cc:240] Negotiation successful
0926 16:32:14.968917 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":43,"thread_start_us":83,"threads_started":1}
I20260926 16:32:14.969337 13469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60626cc751cd4e32a766895981a084aa" candidate_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" is_pre_election: true
I20260926 16:32:14.969678 13469 raft_consensus.cc:2454] T 60626cc751cd4e32a766895981a084aa P 0003ec5f5cdd45f291cc2505083bcd8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ebf7b662b0724a89b0f660920b5f4e8e in term 0.
I20260926 16:32:14.969679 13189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60626cc751cd4e32a766895981a084aa" candidate_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c2dc32c75904237a8d40ae792d3c57e" is_pre_election: true
I20260926 16:32:14.970026 13189 raft_consensus.cc:2454] T 60626cc751cd4e32a766895981a084aa P 8c2dc32c75904237a8d40ae792d3c57e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ebf7b662b0724a89b0f660920b5f4e8e in term 0.
I20260926 16:32:14.970233 13263 leader_election.cc:302] T 60626cc751cd4e32a766895981a084aa P ebf7b662b0724a89b0f660920b5f4e8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0003ec5f5cdd45f291cc2505083bcd8e, ebf7b662b0724a89b0f660920b5f4e8e; no voters:
I20260926 16:32:14.970569 13544 raft_consensus.cc:2781] T 60626cc751cd4e32a766895981a084aa P ebf7b662b0724a89b0f660920b5f4e8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:32:14.970652 13544 raft_consensus.cc:482] T 60626cc751cd4e32a766895981a084aa P ebf7b662b0724a89b0f660920b5f4e8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:32:14.970705 13544 raft_consensus.cc:3037] T 60626cc751cd4e32a766895981a084aa P ebf7b662b0724a89b0f660920b5f4e8e [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:14.971729 13544 raft_consensus.cc:504] T 60626cc751cd4e32a766895981a084aa P ebf7b662b0724a89b0f660920b5f4e8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:14.972059 13544 leader_election.cc:288] T 60626cc751cd4e32a766895981a084aa P ebf7b662b0724a89b0f660920b5f4e8e [CANDIDATE]: Term 1 election: Requested vote from peers 8c2dc32c75904237a8d40ae792d3c57e (127.11.8.193:33355), 0003ec5f5cdd45f291cc2505083bcd8e (127.11.8.195:37479)
I20260926 16:32:14.972388 13189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60626cc751cd4e32a766895981a084aa" candidate_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c2dc32c75904237a8d40ae792d3c57e"
I20260926 16:32:14.972409 13469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60626cc751cd4e32a766895981a084aa" candidate_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0003ec5f5cdd45f291cc2505083bcd8e"
I20260926 16:32:14.972532 13189 raft_consensus.cc:3037] T 60626cc751cd4e32a766895981a084aa P 8c2dc32c75904237a8d40ae792d3c57e [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:14.972543 13469 raft_consensus.cc:3037] T 60626cc751cd4e32a766895981a084aa P 0003ec5f5cdd45f291cc2505083bcd8e [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:14.973623 13469 raft_consensus.cc:2454] T 60626cc751cd4e32a766895981a084aa P 0003ec5f5cdd45f291cc2505083bcd8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebf7b662b0724a89b0f660920b5f4e8e in term 1.
I20260926 16:32:14.973639 13189 raft_consensus.cc:2454] T 60626cc751cd4e32a766895981a084aa P 8c2dc32c75904237a8d40ae792d3c57e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebf7b662b0724a89b0f660920b5f4e8e in term 1.
I20260926 16:32:14.973934 13263 leader_election.cc:302] T 60626cc751cd4e32a766895981a084aa P ebf7b662b0724a89b0f660920b5f4e8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0003ec5f5cdd45f291cc2505083bcd8e, ebf7b662b0724a89b0f660920b5f4e8e; no voters:
I20260926 16:32:14.974134 13544 raft_consensus.cc:2781] T 60626cc751cd4e32a766895981a084aa P ebf7b662b0724a89b0f660920b5f4e8e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:32:14.974399 13544 raft_consensus.cc:686] T 60626cc751cd4e32a766895981a084aa P ebf7b662b0724a89b0f660920b5f4e8e [term 1 LEADER]: Becoming Leader. State: Replica: ebf7b662b0724a89b0f660920b5f4e8e, State: Running, Role: LEADER
I20260926 16:32:14.975528 13544 consensus_queue.cc:237] T 60626cc751cd4e32a766895981a084aa P ebf7b662b0724a89b0f660920b5f4e8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
W20260926 16:32:14.978207 13516 tablet.cc:2369] T 60626cc751cd4e32a766895981a084aa P 0003ec5f5cdd45f291cc2505083bcd8e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:32:14.979435 13030 catalog_manager.cc:5495] T 60626cc751cd4e32a766895981a084aa P ebf7b662b0724a89b0f660920b5f4e8e reported cstate change: term changed from 0 to 1, leader changed from <none> to ebf7b662b0724a89b0f660920b5f4e8e (127.11.8.194). New cstate: current_term: 1 leader_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } health_report { overall_health: UNKNOWN } } }
W20260926 16:32:15.004346 13376 tablet.cc:2369] T 60626cc751cd4e32a766895981a084aa P ebf7b662b0724a89b0f660920b5f4e8e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 16:32:15.026032 13236 tablet.cc:2369] T 60626cc751cd4e32a766895981a084aa P 8c2dc32c75904237a8d40ae792d3c57e: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 26 16:32:15 dist-test-slave-2zvq krb5kdc[12987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440333, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:32:15.052841 13555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:15.043997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:34553 (local address 127.0.0.1:45824)
0926 16:32:15.044277 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:15.044289 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:15.044411 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:15.044647 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:15.044649 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:15.045112 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:15.045204 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:15.045210 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:15.045948 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:15.045951 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:15.046633 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:15.046642 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:15.047061 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:15.047064 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:15.047126 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:15.047661 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:15.047673 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:15.049191 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:15.051069 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:15.051077 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:15.051081 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:15.051425 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:15.051725 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:15.051727 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:15.051729 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:15.051841 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:15.052310 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:15.052316 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:15.052468 (+ 152us) client_negotiation.cc:769] Sending connection context
0926 16:32:15.052562 (+ 94us) client_negotiation.cc:240] Negotiation successful
0926 16:32:15.052678 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":94,"threads_started":1}
I20260926 16:32:15.052855 13081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:15.044102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45824 (local address 127.11.8.254:34553)
0926 16:32:15.044288 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:15.044292 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:15.044380 (+ 88us) server_negotiation.cc:407] Connection header received
0926 16:32:15.044492 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:15.044494 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:15.044542 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:15.044621 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:15.045315 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:15.045841 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:15.046744 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:15.046962 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:15.047048 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:15.049349 (+ 2301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:15.049352 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:15.049354 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:15.049375 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:15.050950 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:15.051541 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:15.051545 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:15.051546 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:15.051626 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:15.051988 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:15.051993 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:15.051995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:15.052199 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:15.052321 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:15.052632 (+ 311us) server_negotiation.cc:299] Negotiation successful
0926 16:32:15.052732 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260926 16:32:15.057034 13030 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:45824:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 16:32:15.057855 13030 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:32:15.062989 13449 tablet_service.cc:1467] Processing CreateTablet for tablet 28ec4050da464f1682b3de942c12eece (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9cdb57d6b2b84db3a1f6beaaaa712b32]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 16:32:15.062824 13169 tablet_service.cc:1467] Processing CreateTablet for tablet 28ec4050da464f1682b3de942c12eece (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9cdb57d6b2b84db3a1f6beaaaa712b32]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 16:32:15.062855 13309 tablet_service.cc:1467] Processing CreateTablet for tablet 28ec4050da464f1682b3de942c12eece (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9cdb57d6b2b84db3a1f6beaaaa712b32]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 16:32:15.063601 13309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28ec4050da464f1682b3de942c12eece. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:15.063601 13169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28ec4050da464f1682b3de942c12eece. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:15.063612 13449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28ec4050da464f1682b3de942c12eece. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:15.066006 13540 tablet_bootstrap.cc:492] T 28ec4050da464f1682b3de942c12eece P 0003ec5f5cdd45f291cc2505083bcd8e: Bootstrap starting.
I20260926 16:32:15.066447 13539 tablet_bootstrap.cc:492] T 28ec4050da464f1682b3de942c12eece P 8c2dc32c75904237a8d40ae792d3c57e: Bootstrap starting.
I20260926 16:32:15.066584 13538 tablet_bootstrap.cc:492] T 28ec4050da464f1682b3de942c12eece P ebf7b662b0724a89b0f660920b5f4e8e: Bootstrap starting.
I20260926 16:32:15.067102 13540 tablet_bootstrap.cc:654] T 28ec4050da464f1682b3de942c12eece P 0003ec5f5cdd45f291cc2505083bcd8e: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:15.067413 13538 tablet_bootstrap.cc:654] T 28ec4050da464f1682b3de942c12eece P ebf7b662b0724a89b0f660920b5f4e8e: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:15.067458 13539 tablet_bootstrap.cc:654] T 28ec4050da464f1682b3de942c12eece P 8c2dc32c75904237a8d40ae792d3c57e: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:15.068394 13540 tablet_bootstrap.cc:492] T 28ec4050da464f1682b3de942c12eece P 0003ec5f5cdd45f291cc2505083bcd8e: No bootstrap required, opened a new log
I20260926 16:32:15.068476 13540 ts_tablet_manager.cc:1397] T 28ec4050da464f1682b3de942c12eece P 0003ec5f5cdd45f291cc2505083bcd8e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260926 16:32:15.068936 13539 tablet_bootstrap.cc:492] T 28ec4050da464f1682b3de942c12eece P 8c2dc32c75904237a8d40ae792d3c57e: No bootstrap required, opened a new log
I20260926 16:32:15.068956 13540 raft_consensus.cc:348] T 28ec4050da464f1682b3de942c12eece P 0003ec5f5cdd45f291cc2505083bcd8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:15.069037 13539 ts_tablet_manager.cc:1397] T 28ec4050da464f1682b3de942c12eece P 8c2dc32c75904237a8d40ae792d3c57e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260926 16:32:15.069075 13540 raft_consensus.cc:374] T 28ec4050da464f1682b3de942c12eece P 0003ec5f5cdd45f291cc2505083bcd8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:15.069118 13540 raft_consensus.cc:729] T 28ec4050da464f1682b3de942c12eece P 0003ec5f5cdd45f291cc2505083bcd8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0003ec5f5cdd45f291cc2505083bcd8e, State: Initialized, Role: FOLLOWER
I20260926 16:32:15.069167 13538 tablet_bootstrap.cc:492] T 28ec4050da464f1682b3de942c12eece P ebf7b662b0724a89b0f660920b5f4e8e: No bootstrap required, opened a new log
I20260926 16:32:15.069235 13538 ts_tablet_manager.cc:1397] T 28ec4050da464f1682b3de942c12eece P ebf7b662b0724a89b0f660920b5f4e8e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260926 16:32:15.069236 13540 consensus_queue.cc:260] T 28ec4050da464f1682b3de942c12eece P 0003ec5f5cdd45f291cc2505083bcd8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:15.069495 13540 ts_tablet_manager.cc:1428] T 28ec4050da464f1682b3de942c12eece P 0003ec5f5cdd45f291cc2505083bcd8e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 16:32:15.069521 13539 raft_consensus.cc:348] T 28ec4050da464f1682b3de942c12eece P 8c2dc32c75904237a8d40ae792d3c57e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:15.069620 13539 raft_consensus.cc:374] T 28ec4050da464f1682b3de942c12eece P 8c2dc32c75904237a8d40ae792d3c57e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:15.069659 13539 raft_consensus.cc:729] T 28ec4050da464f1682b3de942c12eece P 8c2dc32c75904237a8d40ae792d3c57e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c2dc32c75904237a8d40ae792d3c57e, State: Initialized, Role: FOLLOWER
I20260926 16:32:15.069674 13538 raft_consensus.cc:348] T 28ec4050da464f1682b3de942c12eece P ebf7b662b0724a89b0f660920b5f4e8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:15.069787 13538 raft_consensus.cc:374] T 28ec4050da464f1682b3de942c12eece P ebf7b662b0724a89b0f660920b5f4e8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:15.069765 13539 consensus_queue.cc:260] T 28ec4050da464f1682b3de942c12eece P 8c2dc32c75904237a8d40ae792d3c57e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:15.069831 13538 raft_consensus.cc:729] T 28ec4050da464f1682b3de942c12eece P ebf7b662b0724a89b0f660920b5f4e8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebf7b662b0724a89b0f660920b5f4e8e, State: Initialized, Role: FOLLOWER
I20260926 16:32:15.069962 13538 consensus_queue.cc:260] T 28ec4050da464f1682b3de942c12eece P ebf7b662b0724a89b0f660920b5f4e8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:15.070029 13539 ts_tablet_manager.cc:1428] T 28ec4050da464f1682b3de942c12eece P 8c2dc32c75904237a8d40ae792d3c57e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260926 16:32:15.069502 13546 tablet_replica.cc:439] T 28ec4050da464f1682b3de942c12eece P 0003ec5f5cdd45f291cc2505083bcd8e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } } }
I20260926 16:32:15.070223 13538 ts_tablet_manager.cc:1428] T 28ec4050da464f1682b3de942c12eece P ebf7b662b0724a89b0f660920b5f4e8e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 16:32:15.070298 13546 tablet_replica.cc:442] T 28ec4050da464f1682b3de942c12eece P 0003ec5f5cdd45f291cc2505083bcd8e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 16:32:15.070284 13544 tablet_replica.cc:439] T 28ec4050da464f1682b3de942c12eece P ebf7b662b0724a89b0f660920b5f4e8e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } } }
I20260926 16:32:15.070407 13544 tablet_replica.cc:442] T 28ec4050da464f1682b3de942c12eece P ebf7b662b0724a89b0f660920b5f4e8e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 16:32:15.070010 13545 tablet_replica.cc:439] T 28ec4050da464f1682b3de942c12eece P 8c2dc32c75904237a8d40ae792d3c57e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } } }
I20260926 16:32:15.071025 13545 tablet_replica.cc:442] T 28ec4050da464f1682b3de942c12eece P 8c2dc32c75904237a8d40ae792d3c57e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 16:32:15.245461 13546 raft_consensus.cc:482] T 28ec4050da464f1682b3de942c12eece P 0003ec5f5cdd45f291cc2505083bcd8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:32:15.245616 13546 raft_consensus.cc:504] T 28ec4050da464f1682b3de942c12eece P 0003ec5f5cdd45f291cc2505083bcd8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:15.246979 13546 leader_election.cc:288] T 28ec4050da464f1682b3de942c12eece P 0003ec5f5cdd45f291cc2505083bcd8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c2dc32c75904237a8d40ae792d3c57e (127.11.8.193:33355), ebf7b662b0724a89b0f660920b5f4e8e (127.11.8.194:43943)
I20260926 16:32:15.250903 13535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:15.247497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:38295 (local address 127.11.8.193:33355)
0926 16:32:15.247619 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:15.247624 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:15.247638 (+ 14us) server_negotiation.cc:407] Connection header received
0926 16:32:15.247792 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:15.247795 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:15.247836 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:15.247952 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:15.248214 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:15.248716 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:15.249876 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:15.250259 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:15.250447 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:15.250500 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:15.250691 (+ 191us) server_negotiation.cc:299] Negotiation successful
0926 16:32:15.250764 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 16:32:15.250909 13534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:15.247409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:36241 (local address 127.11.8.194:43943)
0926 16:32:15.247589 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:15.247593 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:15.247698 (+ 105us) server_negotiation.cc:407] Connection header received
0926 16:32:15.247840 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:15.247844 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:15.247894 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:15.247977 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:15.248261 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:15.248770 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:15.249877 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:15.250334 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:15.250464 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:15.250515 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:15.250714 (+ 199us) server_negotiation.cc:299] Negotiation successful
0926 16:32:15.250787 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260926 16:32:15.250924 13559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:15.247204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:33355 (local address 127.11.8.195:38295)
0926 16:32:15.247543 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:15.247559 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:15.247666 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:15.247961 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:15.247964 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:15.247980 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:15.248107 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:15.248112 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:15.248821 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:15.248827 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:15.249741 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:15.249749 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:15.250436 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:15.250439 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:15.250549 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:15.250577 (+ 28us) client_negotiation.cc:769] Sending connection context
0926 16:32:15.250664 (+ 87us) client_negotiation.cc:240] Negotiation successful
0926 16:32:15.250738 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":61,"spinlock_wait_cycles":15232,"thread_start_us":90,"threads_started":1}
I20260926 16:32:15.250936 13560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:15.247204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:43943 (local address 127.11.8.195:36241)
0926 16:32:15.247594 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:15.247609 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:15.247742 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:15.247988 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:15.247991 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:15.248030 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:15.248148 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:15.248153 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:15.248882 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:15.248887 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:15.249746 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:15.249757 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:15.250457 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:15.250460 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:15.250577 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:15.250605 (+ 28us) client_negotiation.cc:769] Sending connection context
0926 16:32:15.250698 (+ 93us) client_negotiation.cc:240] Negotiation successful
0926 16:32:15.250770 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"mutex_wait_us":75,"spinlock_wait_cycles":2048,"thread_start_us":87,"threads_started":1}
I20260926 16:32:15.251502 13189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28ec4050da464f1682b3de942c12eece" candidate_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c2dc32c75904237a8d40ae792d3c57e" is_pre_election: true
I20260926 16:32:15.251691 13189 raft_consensus.cc:2454] T 28ec4050da464f1682b3de942c12eece P 8c2dc32c75904237a8d40ae792d3c57e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0003ec5f5cdd45f291cc2505083bcd8e in term 0.
I20260926 16:32:15.251801 13329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28ec4050da464f1682b3de942c12eece" candidate_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" is_pre_election: true
I20260926 16:32:15.252033 13329 raft_consensus.cc:2454] T 28ec4050da464f1682b3de942c12eece P ebf7b662b0724a89b0f660920b5f4e8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0003ec5f5cdd45f291cc2505083bcd8e in term 0.
I20260926 16:32:15.252110 13402 leader_election.cc:302] T 28ec4050da464f1682b3de942c12eece P 0003ec5f5cdd45f291cc2505083bcd8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0003ec5f5cdd45f291cc2505083bcd8e, 8c2dc32c75904237a8d40ae792d3c57e; no voters:
I20260926 16:32:15.252429 13546 raft_consensus.cc:2781] T 28ec4050da464f1682b3de942c12eece P 0003ec5f5cdd45f291cc2505083bcd8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:32:15.252514 13546 raft_consensus.cc:482] T 28ec4050da464f1682b3de942c12eece P 0003ec5f5cdd45f291cc2505083bcd8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:32:15.252561 13546 raft_consensus.cc:3037] T 28ec4050da464f1682b3de942c12eece P 0003ec5f5cdd45f291cc2505083bcd8e [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:15.253645 13546 raft_consensus.cc:504] T 28ec4050da464f1682b3de942c12eece P 0003ec5f5cdd45f291cc2505083bcd8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:15.254007 13546 leader_election.cc:288] T 28ec4050da464f1682b3de942c12eece P 0003ec5f5cdd45f291cc2505083bcd8e [CANDIDATE]: Term 1 election: Requested vote from peers 8c2dc32c75904237a8d40ae792d3c57e (127.11.8.193:33355), ebf7b662b0724a89b0f660920b5f4e8e (127.11.8.194:43943)
I20260926 16:32:15.254408 13189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28ec4050da464f1682b3de942c12eece" candidate_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c2dc32c75904237a8d40ae792d3c57e"
I20260926 16:32:15.254545 13189 raft_consensus.cc:3037] T 28ec4050da464f1682b3de942c12eece P 8c2dc32c75904237a8d40ae792d3c57e [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:15.254521 13329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28ec4050da464f1682b3de942c12eece" candidate_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebf7b662b0724a89b0f660920b5f4e8e"
I20260926 16:32:15.254676 13329 raft_consensus.cc:3037] T 28ec4050da464f1682b3de942c12eece P ebf7b662b0724a89b0f660920b5f4e8e [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:15.255499 13189 raft_consensus.cc:2454] T 28ec4050da464f1682b3de942c12eece P 8c2dc32c75904237a8d40ae792d3c57e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0003ec5f5cdd45f291cc2505083bcd8e in term 1.
I20260926 16:32:15.255766 13329 raft_consensus.cc:2454] T 28ec4050da464f1682b3de942c12eece P ebf7b662b0724a89b0f660920b5f4e8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0003ec5f5cdd45f291cc2505083bcd8e in term 1.
I20260926 16:32:15.255777 13402 leader_election.cc:302] T 28ec4050da464f1682b3de942c12eece P 0003ec5f5cdd45f291cc2505083bcd8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0003ec5f5cdd45f291cc2505083bcd8e, 8c2dc32c75904237a8d40ae792d3c57e; no voters:
I20260926 16:32:15.255966 13546 raft_consensus.cc:2781] T 28ec4050da464f1682b3de942c12eece P 0003ec5f5cdd45f291cc2505083bcd8e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:32:15.256198 13546 raft_consensus.cc:686] T 28ec4050da464f1682b3de942c12eece P 0003ec5f5cdd45f291cc2505083bcd8e [term 1 LEADER]: Becoming Leader. State: Replica: 0003ec5f5cdd45f291cc2505083bcd8e, State: Running, Role: LEADER
I20260926 16:32:15.256532 13546 consensus_queue.cc:237] T 28ec4050da464f1682b3de942c12eece P 0003ec5f5cdd45f291cc2505083bcd8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:15.258783 13561 tablet_replica.cc:439] T 28ec4050da464f1682b3de942c12eece P 0003ec5f5cdd45f291cc2505083bcd8e: TxnStatusTablet state changed. Reason: New leader 0003ec5f5cdd45f291cc2505083bcd8e. Latest consensus state: current_term: 1 leader_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } } }
I20260926 16:32:15.258898 13561 tablet_replica.cc:442] T 28ec4050da464f1682b3de942c12eece P 0003ec5f5cdd45f291cc2505083bcd8e: This TxnStatusTablet replica's current role is: LEADER
I20260926 16:32:15.259204 13563 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 16:32:15.259411 13563 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 16:32:15.260090 13032 catalog_manager.cc:5495] T 28ec4050da464f1682b3de942c12eece P 0003ec5f5cdd45f291cc2505083bcd8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0003ec5f5cdd45f291cc2505083bcd8e (127.11.8.195). New cstate: current_term: 1 leader_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } health_report { overall_health: HEALTHY } } }
I20260926 16:32:15.314046 13555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:15.310115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:37479 (local address 127.0.0.1:49092)
0926 16:32:15.310314 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:15.310332 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:15.310441 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:15.310769 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:15.310771 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:15.310779 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 16:32:15.310864 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:15.310868 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:15.311825 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:15.311827 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:15.312409 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:15.312416 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:15.312978 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:15.312981 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:15.313047 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:15.313087 (+ 40us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 16:32:15.313811 (+ 724us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 16:32:15.313821 (+ 10us) client_negotiation.cc:769] Sending connection context
0926 16:32:15.313873 (+ 52us) client_negotiation.cc:240] Negotiation successful
0926 16:32:15.313936 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260926 16:32:15.314069 13537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:15.310248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49092 (local address 127.11.8.195:37479)
0926 16:32:15.310425 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:15.310430 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:15.310447 (+ 17us) server_negotiation.cc:407] Connection header received
0926 16:32:15.310578 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:15.310581 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:15.310648 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:15.310757 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:32:15.310954 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:15.311717 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:15.312526 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:15.312871 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:15.312965 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:15.313215 (+ 250us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:32:15.313710 (+ 495us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:32:15.313834 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:15.313888 (+ 54us) server_negotiation.cc:299] Negotiation successful
0926 16:32:15.313937 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260926 16:32:15.319375 13329 raft_consensus.cc:1261] T 28ec4050da464f1682b3de942c12eece P ebf7b662b0724a89b0f660920b5f4e8e [term 1 FOLLOWER]: Refusing update from remote peer 0003ec5f5cdd45f291cc2505083bcd8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:32:15.319375 13189 raft_consensus.cc:1261] T 28ec4050da464f1682b3de942c12eece P 8c2dc32c75904237a8d40ae792d3c57e [term 1 FOLLOWER]: Refusing update from remote peer 0003ec5f5cdd45f291cc2505083bcd8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:32:15.320024 13546 consensus_queue.cc:1035] T 28ec4050da464f1682b3de942c12eece P 0003ec5f5cdd45f291cc2505083bcd8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:32:15.320253 13561 consensus_queue.cc:1035] T 28ec4050da464f1682b3de942c12eece P 0003ec5f5cdd45f291cc2505083bcd8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:32:15.324601 13544 tablet_replica.cc:439] T 28ec4050da464f1682b3de942c12eece P ebf7b662b0724a89b0f660920b5f4e8e: TxnStatusTablet state changed. Reason: New leader 0003ec5f5cdd45f291cc2505083bcd8e. Latest consensus state: current_term: 1 leader_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } } }
I20260926 16:32:15.324726 13544 tablet_replica.cc:442] T 28ec4050da464f1682b3de942c12eece P ebf7b662b0724a89b0f660920b5f4e8e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 16:32:15.325591 13545 tablet_replica.cc:439] T 28ec4050da464f1682b3de942c12eece P 8c2dc32c75904237a8d40ae792d3c57e: TxnStatusTablet state changed. Reason: New leader 0003ec5f5cdd45f291cc2505083bcd8e. Latest consensus state: current_term: 1 leader_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } } }
I20260926 16:32:15.325711 13545 tablet_replica.cc:442] T 28ec4050da464f1682b3de942c12eece P 8c2dc32c75904237a8d40ae792d3c57e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 16:32:15.327462 13561 tablet_replica.cc:439] T 28ec4050da464f1682b3de942c12eece P 0003ec5f5cdd45f291cc2505083bcd8e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } } }
I20260926 16:32:15.327576 13561 tablet_replica.cc:442] T 28ec4050da464f1682b3de942c12eece P 0003ec5f5cdd45f291cc2505083bcd8e: This TxnStatusTablet replica's current role is: LEADER
I20260926 16:32:15.327679 13566 mvcc.cc:205] Tried to move back new op lower bound from 7333643613461684224 to 7333643613213241344. Current Snapshot: MvccSnapshot[applied={T|T < 7333643613461684224}]
I20260926 16:32:15.328890 13546 tablet_replica.cc:439] T 28ec4050da464f1682b3de942c12eece P 0003ec5f5cdd45f291cc2505083bcd8e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } } }
I20260926 16:32:15.329010 13546 tablet_replica.cc:442] T 28ec4050da464f1682b3de942c12eece P 0003ec5f5cdd45f291cc2505083bcd8e: This TxnStatusTablet replica's current role is: LEADER
I20260926 16:32:15.329329 13544 tablet_replica.cc:439] T 28ec4050da464f1682b3de942c12eece P ebf7b662b0724a89b0f660920b5f4e8e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } } }
I20260926 16:32:15.329429 13544 tablet_replica.cc:442] T 28ec4050da464f1682b3de942c12eece P ebf7b662b0724a89b0f660920b5f4e8e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 16:32:15.330334 13567 mvcc.cc:205] Tried to move back new op lower bound from 7333643613461684224 to 7333643613213241344. Current Snapshot: MvccSnapshot[applied={T|T < 7333643613461684224}]
I20260926 16:32:15.331787 13545 tablet_replica.cc:439] T 28ec4050da464f1682b3de942c12eece P 8c2dc32c75904237a8d40ae792d3c57e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } } }
I20260926 16:32:15.331903 13545 tablet_replica.cc:442] T 28ec4050da464f1682b3de942c12eece P 8c2dc32c75904237a8d40ae792d3c57e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 16:32:15.336221 13569 mvcc.cc:205] Tried to move back new op lower bound from 7333643613461684224 to 7333643613213241344. Current Snapshot: MvccSnapshot[applied={T|T < 7333643613461684224}]
I20260926 16:32:15.339792 13534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:15.335458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55684 (local address 127.11.8.194:43943)
0926 16:32:15.335579 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:15.335584 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:15.335743 (+ 159us) server_negotiation.cc:407] Connection header received
0926 16:32:15.335910 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:15.335913 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:15.335988 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:15.336096 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:32:15.336406 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:15.337188 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:15.338213 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:15.338587 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:15.338685 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:15.338928 (+ 243us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:32:15.339425 (+ 497us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:32:15.339537 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:15.339592 (+ 55us) server_negotiation.cc:299] Negotiation successful
0926 16:32:15.339647 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 16:32:15.350409 13537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:15.347138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:52897 (local address 127.11.8.195:37479)
0926 16:32:15.347284 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:15.347289 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:15.347398 (+ 109us) server_negotiation.cc:407] Connection header received
0926 16:32:15.347556 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:15.347559 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:15.347614 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:15.347694 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:32:15.347928 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:15.348431 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:15.349294 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:15.349529 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:15.349644 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:15.349956 (+ 312us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:32:15.350033 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:32:15.350114 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:15.350199 (+ 85us) server_negotiation.cc:299] Negotiation successful
0926 16:32:15.350244 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 16:32:15.350441 13578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:15.347014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:37479 (local address 127.11.8.194:52897)
0926 16:32:15.347315 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:15.347335 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:15.347458 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:15.347703 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:15.347706 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:15.347713 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 16:32:15.347827 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:15.347832 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:15.348540 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:15.348545 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:15.349176 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:15.349184 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:15.349656 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:15.349659 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:15.349746 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:15.349785 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 16:32:15.350117 (+ 332us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 16:32:15.350126 (+ 9us) client_negotiation.cc:769] Sending connection context
0926 16:32:15.350190 (+ 64us) client_negotiation.cc:240] Negotiation successful
0926 16:32:15.350288 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":109,"threads_started":1}
I20260926 16:32:15.357973 13189 raft_consensus.cc:1261] T 60626cc751cd4e32a766895981a084aa P 8c2dc32c75904237a8d40ae792d3c57e [term 1 FOLLOWER]: Refusing update from remote peer ebf7b662b0724a89b0f660920b5f4e8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:32:15.358040 13469 raft_consensus.cc:1261] T 60626cc751cd4e32a766895981a084aa P 0003ec5f5cdd45f291cc2505083bcd8e [term 1 FOLLOWER]: Refusing update from remote peer ebf7b662b0724a89b0f660920b5f4e8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:32:15.358507 13548 consensus_queue.cc:1035] T 60626cc751cd4e32a766895981a084aa P ebf7b662b0724a89b0f660920b5f4e8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:32:15.358891 13548 consensus_queue.cc:1035] T 60626cc751cd4e32a766895981a084aa P ebf7b662b0724a89b0f660920b5f4e8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:32:15.382190 13534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:15.377609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:34725 (local address 127.11.8.194:43943)
0926 16:32:15.377721 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:15.377727 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:15.377974 (+ 247us) server_negotiation.cc:407] Connection header received
0926 16:32:15.378106 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:15.378110 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:15.378169 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:15.378259 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:32:15.379280 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:15.379854 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:15.381047 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:15.381286 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:15.381406 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:15.381700 (+ 294us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:32:15.381780 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:32:15.381858 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:15.381986 (+ 128us) server_negotiation.cc:299] Negotiation successful
0926 16:32:15.382040 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 16:32:15.382208 13584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:15.377514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:43943 (local address 127.11.8.195:34725)
0926 16:32:15.377826 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:15.377839 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:15.377994 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:15.378382 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:15.378386 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:15.378401 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 16:32:15.379125 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:15.379135 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:15.379968 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:15.379974 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:15.380925 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:15.380935 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:15.381415 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:15.381419 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:15.381509 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:15.381563 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 16:32:15.381873 (+ 310us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 16:32:15.381882 (+ 9us) client_negotiation.cc:769] Sending connection context
0926 16:32:15.381971 (+ 89us) client_negotiation.cc:240] Negotiation successful
0926 16:32:15.382056 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":112,"threads_started":1}
I20260926 16:32:15.406663 13032 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:32:15.406886 13032 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:32:15.409713 13032 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d59182d471a4c5596b65f20feab477e: Sending DeleteTablet for 3 replicas of tablet 60626cc751cd4e32a766895981a084aa
I20260926 16:32:15.410444 13309 tablet_service.cc:1514] Processing DeleteTablet for tablet 60626cc751cd4e32a766895981a084aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:32:15 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:55670
I20260926 16:32:15.410710 13169 tablet_service.cc:1514] Processing DeleteTablet for tablet 60626cc751cd4e32a766895981a084aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:32:15 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:54260
I20260926 16:32:15.411113 13587 tablet_replica.cc:330] T 60626cc751cd4e32a766895981a084aa P ebf7b662b0724a89b0f660920b5f4e8e: stopping tablet replica
I20260926 16:32:15.411456 13587 raft_consensus.cc:2229] T 60626cc751cd4e32a766895981a084aa P ebf7b662b0724a89b0f660920b5f4e8e [term 1 LEADER]: Raft consensus shutting down.
I20260926 16:32:15.411518 13449 tablet_service.cc:1514] Processing DeleteTablet for tablet 60626cc751cd4e32a766895981a084aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:32:15 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:49088
I20260926 16:32:15.411931 13587 raft_consensus.cc:2258] T 60626cc751cd4e32a766895981a084aa P ebf7b662b0724a89b0f660920b5f4e8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:32:15.412837 13032 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45808:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:32:15.413316 13032 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:32:15.413677 13587 ts_tablet_manager.cc:1905] T 60626cc751cd4e32a766895981a084aa P ebf7b662b0724a89b0f660920b5f4e8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:32:15.415283 13589 tablet_replica.cc:330] T 60626cc751cd4e32a766895981a084aa P 0003ec5f5cdd45f291cc2505083bcd8e: stopping tablet replica
I20260926 16:32:15.415634 13589 raft_consensus.cc:2229] T 60626cc751cd4e32a766895981a084aa P 0003ec5f5cdd45f291cc2505083bcd8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:32:15.415835 13589 raft_consensus.cc:2258] T 60626cc751cd4e32a766895981a084aa P 0003ec5f5cdd45f291cc2505083bcd8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:32:15.416208 13588 tablet_replica.cc:330] T 60626cc751cd4e32a766895981a084aa P 8c2dc32c75904237a8d40ae792d3c57e: stopping tablet replica
I20260926 16:32:15.416504 13588 raft_consensus.cc:2229] T 60626cc751cd4e32a766895981a084aa P 8c2dc32c75904237a8d40ae792d3c57e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:32:15.416700 13588 raft_consensus.cc:2258] T 60626cc751cd4e32a766895981a084aa P 8c2dc32c75904237a8d40ae792d3c57e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:32:15.419466 13589 ts_tablet_manager.cc:1905] T 60626cc751cd4e32a766895981a084aa P 0003ec5f5cdd45f291cc2505083bcd8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:32:15.420073 13169 tablet_service.cc:1467] Processing CreateTablet for tablet fd80bf2d9e344d989051b5324742d837 (DEFAULT_TABLE table=test-table [id=56ed8458042f41748e4199c8ee061f0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:15.420405 13169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd80bf2d9e344d989051b5324742d837. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:15.422099 13588 ts_tablet_manager.cc:1905] T 60626cc751cd4e32a766895981a084aa P 8c2dc32c75904237a8d40ae792d3c57e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:32:15.422552 13309 tablet_service.cc:1467] Processing CreateTablet for tablet fd80bf2d9e344d989051b5324742d837 (DEFAULT_TABLE table=test-table [id=56ed8458042f41748e4199c8ee061f0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:15.422811 13309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd80bf2d9e344d989051b5324742d837. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:15.424944 13588 ts_tablet_manager.cc:1918] T 60626cc751cd4e32a766895981a084aa P 8c2dc32c75904237a8d40ae792d3c57e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 16:32:15.425024 13588 log.cc:1196] T 60626cc751cd4e32a766895981a084aa P 8c2dc32c75904237a8d40ae792d3c57e: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-0/wal/wals/60626cc751cd4e32a766895981a084aa
I20260926 16:32:15.425278 13538 tablet_bootstrap.cc:492] T fd80bf2d9e344d989051b5324742d837 P ebf7b662b0724a89b0f660920b5f4e8e: Bootstrap starting.
I20260926 16:32:15.425324 13588 ts_tablet_manager.cc:1939] T 60626cc751cd4e32a766895981a084aa P 8c2dc32c75904237a8d40ae792d3c57e: Deleting consensus metadata
I20260926 16:32:15.426199 13538 tablet_bootstrap.cc:654] T fd80bf2d9e344d989051b5324742d837 P ebf7b662b0724a89b0f660920b5f4e8e: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:15.427337 13018 catalog_manager.cc:4841] TS 8c2dc32c75904237a8d40ae792d3c57e (127.11.8.193:33355): tablet 60626cc751cd4e32a766895981a084aa (table test-table [id=b0dc740d03b74ee78d17d48459b1669d]) successfully deleted
I20260926 16:32:15.428035 13589 ts_tablet_manager.cc:1918] T 60626cc751cd4e32a766895981a084aa P 0003ec5f5cdd45f291cc2505083bcd8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 16:32:15.428125 13589 log.cc:1196] T 60626cc751cd4e32a766895981a084aa P 0003ec5f5cdd45f291cc2505083bcd8e: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-2/wal/wals/60626cc751cd4e32a766895981a084aa
I20260926 16:32:15.428306 13449 tablet_service.cc:1467] Processing CreateTablet for tablet fd80bf2d9e344d989051b5324742d837 (DEFAULT_TABLE table=test-table [id=56ed8458042f41748e4199c8ee061f0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:15.428426 13589 ts_tablet_manager.cc:1939] T 60626cc751cd4e32a766895981a084aa P 0003ec5f5cdd45f291cc2505083bcd8e: Deleting consensus metadata
I20260926 16:32:15.428544 13449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd80bf2d9e344d989051b5324742d837. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:15.430537 13019 catalog_manager.cc:4841] TS 0003ec5f5cdd45f291cc2505083bcd8e (127.11.8.195:37479): tablet 60626cc751cd4e32a766895981a084aa (table test-table [id=b0dc740d03b74ee78d17d48459b1669d]) successfully deleted
I20260926 16:32:15.430667 13540 tablet_bootstrap.cc:492] T fd80bf2d9e344d989051b5324742d837 P 0003ec5f5cdd45f291cc2505083bcd8e: Bootstrap starting.
I20260926 16:32:15.431542 13540 tablet_bootstrap.cc:654] T fd80bf2d9e344d989051b5324742d837 P 0003ec5f5cdd45f291cc2505083bcd8e: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:15.432094 13539 tablet_bootstrap.cc:492] T fd80bf2d9e344d989051b5324742d837 P 8c2dc32c75904237a8d40ae792d3c57e: Bootstrap starting.
I20260926 16:32:15.433005 13539 tablet_bootstrap.cc:654] T fd80bf2d9e344d989051b5324742d837 P 8c2dc32c75904237a8d40ae792d3c57e: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:15.434252 13587 ts_tablet_manager.cc:1918] T 60626cc751cd4e32a766895981a084aa P ebf7b662b0724a89b0f660920b5f4e8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 16:32:15.434401 13587 log.cc:1196] T 60626cc751cd4e32a766895981a084aa P ebf7b662b0724a89b0f660920b5f4e8e: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-1/wal/wals/60626cc751cd4e32a766895981a084aa
I20260926 16:32:15.434846 13587 ts_tablet_manager.cc:1939] T 60626cc751cd4e32a766895981a084aa P ebf7b662b0724a89b0f660920b5f4e8e: Deleting consensus metadata
I20260926 16:32:15.435436 13539 tablet_bootstrap.cc:492] T fd80bf2d9e344d989051b5324742d837 P 8c2dc32c75904237a8d40ae792d3c57e: No bootstrap required, opened a new log
I20260926 16:32:15.435524 13539 ts_tablet_manager.cc:1397] T fd80bf2d9e344d989051b5324742d837 P 8c2dc32c75904237a8d40ae792d3c57e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260926 16:32:15.435912 13538 tablet_bootstrap.cc:492] T fd80bf2d9e344d989051b5324742d837 P ebf7b662b0724a89b0f660920b5f4e8e: No bootstrap required, opened a new log
I20260926 16:32:15.435986 13538 ts_tablet_manager.cc:1397] T fd80bf2d9e344d989051b5324742d837 P ebf7b662b0724a89b0f660920b5f4e8e: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.001s
I20260926 16:32:15.435946 13539 raft_consensus.cc:348] T fd80bf2d9e344d989051b5324742d837 P 8c2dc32c75904237a8d40ae792d3c57e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:15.436033 13539 raft_consensus.cc:374] T fd80bf2d9e344d989051b5324742d837 P 8c2dc32c75904237a8d40ae792d3c57e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:15.436054 13539 raft_consensus.cc:729] T fd80bf2d9e344d989051b5324742d837 P 8c2dc32c75904237a8d40ae792d3c57e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c2dc32c75904237a8d40ae792d3c57e, State: Initialized, Role: FOLLOWER
I20260926 16:32:15.436187 13539 consensus_queue.cc:260] T fd80bf2d9e344d989051b5324742d837 P 8c2dc32c75904237a8d40ae792d3c57e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:15.436381 13538 raft_consensus.cc:348] T fd80bf2d9e344d989051b5324742d837 P ebf7b662b0724a89b0f660920b5f4e8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:15.436466 13539 ts_tablet_manager.cc:1428] T fd80bf2d9e344d989051b5324742d837 P 8c2dc32c75904237a8d40ae792d3c57e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 16:32:15.436480 13538 raft_consensus.cc:374] T fd80bf2d9e344d989051b5324742d837 P ebf7b662b0724a89b0f660920b5f4e8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:15.436510 13538 raft_consensus.cc:729] T fd80bf2d9e344d989051b5324742d837 P ebf7b662b0724a89b0f660920b5f4e8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebf7b662b0724a89b0f660920b5f4e8e, State: Initialized, Role: FOLLOWER
I20260926 16:32:15.436655 13538 consensus_queue.cc:260] T fd80bf2d9e344d989051b5324742d837 P ebf7b662b0724a89b0f660920b5f4e8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:15.436920 13538 ts_tablet_manager.cc:1428] T fd80bf2d9e344d989051b5324742d837 P ebf7b662b0724a89b0f660920b5f4e8e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260926 16:32:15.438088 13016 catalog_manager.cc:4841] TS ebf7b662b0724a89b0f660920b5f4e8e (127.11.8.194:43943): tablet 60626cc751cd4e32a766895981a084aa (table test-table [id=b0dc740d03b74ee78d17d48459b1669d]) successfully deleted
I20260926 16:32:15.438834 13540 tablet_bootstrap.cc:492] T fd80bf2d9e344d989051b5324742d837 P 0003ec5f5cdd45f291cc2505083bcd8e: No bootstrap required, opened a new log
I20260926 16:32:15.438925 13540 ts_tablet_manager.cc:1397] T fd80bf2d9e344d989051b5324742d837 P 0003ec5f5cdd45f291cc2505083bcd8e: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260926 16:32:15.439335 13540 raft_consensus.cc:348] T fd80bf2d9e344d989051b5324742d837 P 0003ec5f5cdd45f291cc2505083bcd8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:15.439471 13540 raft_consensus.cc:374] T fd80bf2d9e344d989051b5324742d837 P 0003ec5f5cdd45f291cc2505083bcd8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:15.439522 13540 raft_consensus.cc:729] T fd80bf2d9e344d989051b5324742d837 P 0003ec5f5cdd45f291cc2505083bcd8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0003ec5f5cdd45f291cc2505083bcd8e, State: Initialized, Role: FOLLOWER
I20260926 16:32:15.439658 13540 consensus_queue.cc:260] T fd80bf2d9e344d989051b5324742d837 P 0003ec5f5cdd45f291cc2505083bcd8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:15.439994 13540 ts_tablet_manager.cc:1428] T fd80bf2d9e344d989051b5324742d837 P 0003ec5f5cdd45f291cc2505083bcd8e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260926 16:32:15.569770 13582 raft_consensus.cc:482] T fd80bf2d9e344d989051b5324742d837 P ebf7b662b0724a89b0f660920b5f4e8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:32:15.569896 13582 raft_consensus.cc:504] T fd80bf2d9e344d989051b5324742d837 P ebf7b662b0724a89b0f660920b5f4e8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:15.570449 13582 leader_election.cc:288] T fd80bf2d9e344d989051b5324742d837 P ebf7b662b0724a89b0f660920b5f4e8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c2dc32c75904237a8d40ae792d3c57e (127.11.8.193:33355), 0003ec5f5cdd45f291cc2505083bcd8e (127.11.8.195:37479)
I20260926 16:32:15.570927 13189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd80bf2d9e344d989051b5324742d837" candidate_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c2dc32c75904237a8d40ae792d3c57e" is_pre_election: true
I20260926 16:32:15.571003 13469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd80bf2d9e344d989051b5324742d837" candidate_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" is_pre_election: true
I20260926 16:32:15.571161 13469 raft_consensus.cc:2454] T fd80bf2d9e344d989051b5324742d837 P 0003ec5f5cdd45f291cc2505083bcd8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ebf7b662b0724a89b0f660920b5f4e8e in term 0.
I20260926 16:32:15.571252 13189 raft_consensus.cc:2454] T fd80bf2d9e344d989051b5324742d837 P 8c2dc32c75904237a8d40ae792d3c57e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ebf7b662b0724a89b0f660920b5f4e8e in term 0.
I20260926 16:32:15.571516 13263 leader_election.cc:302] T fd80bf2d9e344d989051b5324742d837 P ebf7b662b0724a89b0f660920b5f4e8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0003ec5f5cdd45f291cc2505083bcd8e, ebf7b662b0724a89b0f660920b5f4e8e; no voters:
I20260926 16:32:15.571695 13582 raft_consensus.cc:2781] T fd80bf2d9e344d989051b5324742d837 P ebf7b662b0724a89b0f660920b5f4e8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:32:15.571758 13582 raft_consensus.cc:482] T fd80bf2d9e344d989051b5324742d837 P ebf7b662b0724a89b0f660920b5f4e8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:32:15.571795 13582 raft_consensus.cc:3037] T fd80bf2d9e344d989051b5324742d837 P ebf7b662b0724a89b0f660920b5f4e8e [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:15.572806 13582 raft_consensus.cc:504] T fd80bf2d9e344d989051b5324742d837 P ebf7b662b0724a89b0f660920b5f4e8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:15.573213 13582 leader_election.cc:288] T fd80bf2d9e344d989051b5324742d837 P ebf7b662b0724a89b0f660920b5f4e8e [CANDIDATE]: Term 1 election: Requested vote from peers 8c2dc32c75904237a8d40ae792d3c57e (127.11.8.193:33355), 0003ec5f5cdd45f291cc2505083bcd8e (127.11.8.195:37479)
I20260926 16:32:15.573619 13469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd80bf2d9e344d989051b5324742d837" candidate_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0003ec5f5cdd45f291cc2505083bcd8e"
I20260926 16:32:15.573622 13189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd80bf2d9e344d989051b5324742d837" candidate_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c2dc32c75904237a8d40ae792d3c57e"
I20260926 16:32:15.573756 13189 raft_consensus.cc:3037] T fd80bf2d9e344d989051b5324742d837 P 8c2dc32c75904237a8d40ae792d3c57e [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:15.573784 13469 raft_consensus.cc:3037] T fd80bf2d9e344d989051b5324742d837 P 0003ec5f5cdd45f291cc2505083bcd8e [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:15.574965 13189 raft_consensus.cc:2454] T fd80bf2d9e344d989051b5324742d837 P 8c2dc32c75904237a8d40ae792d3c57e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebf7b662b0724a89b0f660920b5f4e8e in term 1.
I20260926 16:32:15.574994 13469 raft_consensus.cc:2454] T fd80bf2d9e344d989051b5324742d837 P 0003ec5f5cdd45f291cc2505083bcd8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebf7b662b0724a89b0f660920b5f4e8e in term 1.
I20260926 16:32:15.575311 13262 leader_election.cc:302] T fd80bf2d9e344d989051b5324742d837 P ebf7b662b0724a89b0f660920b5f4e8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c2dc32c75904237a8d40ae792d3c57e, ebf7b662b0724a89b0f660920b5f4e8e; no voters:
I20260926 16:32:15.575500 13582 raft_consensus.cc:2781] T fd80bf2d9e344d989051b5324742d837 P ebf7b662b0724a89b0f660920b5f4e8e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:32:15.575608 13582 raft_consensus.cc:686] T fd80bf2d9e344d989051b5324742d837 P ebf7b662b0724a89b0f660920b5f4e8e [term 1 LEADER]: Becoming Leader. State: Replica: ebf7b662b0724a89b0f660920b5f4e8e, State: Running, Role: LEADER
I20260926 16:32:15.575762 13582 consensus_queue.cc:237] T fd80bf2d9e344d989051b5324742d837 P ebf7b662b0724a89b0f660920b5f4e8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:15.577630 13030 catalog_manager.cc:5495] T fd80bf2d9e344d989051b5324742d837 P ebf7b662b0724a89b0f660920b5f4e8e reported cstate change: term changed from 0 to 1, leader changed from <none> to ebf7b662b0724a89b0f660920b5f4e8e (127.11.8.194). New cstate: current_term: 1 leader_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } health_report { overall_health: UNKNOWN } } }
I20260926 16:32:15.597062 13469 raft_consensus.cc:1261] T fd80bf2d9e344d989051b5324742d837 P 0003ec5f5cdd45f291cc2505083bcd8e [term 1 FOLLOWER]: Refusing update from remote peer ebf7b662b0724a89b0f660920b5f4e8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:32:15.597079 13189 raft_consensus.cc:1261] T fd80bf2d9e344d989051b5324742d837 P 8c2dc32c75904237a8d40ae792d3c57e [term 1 FOLLOWER]: Refusing update from remote peer ebf7b662b0724a89b0f660920b5f4e8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:32:15.597447 13582 consensus_queue.cc:1035] T fd80bf2d9e344d989051b5324742d837 P ebf7b662b0724a89b0f660920b5f4e8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:32:15.597612 13544 consensus_queue.cc:1035] T fd80bf2d9e344d989051b5324742d837 P ebf7b662b0724a89b0f660920b5f4e8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 16:32:15.610414 13569 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260926 16:32:17.162253 13032 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:32:17.162464 13032 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:32:17.164549 13032 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d59182d471a4c5596b65f20feab477e: Sending DeleteTablet for 3 replicas of tablet fd80bf2d9e344d989051b5324742d837
I20260926 16:32:17.165124 13169 tablet_service.cc:1514] Processing DeleteTablet for tablet fd80bf2d9e344d989051b5324742d837 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:32:17 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:54260
I20260926 16:32:17.165169 13449 tablet_service.cc:1514] Processing DeleteTablet for tablet fd80bf2d9e344d989051b5324742d837 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:32:17 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:49088
I20260926 16:32:17.165468 13620 tablet_replica.cc:330] T fd80bf2d9e344d989051b5324742d837 P 8c2dc32c75904237a8d40ae792d3c57e: stopping tablet replica
I20260926 16:32:17.165705 13309 tablet_service.cc:1514] Processing DeleteTablet for tablet fd80bf2d9e344d989051b5324742d837 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:32:17 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:55670
I20260926 16:32:17.166046 13620 raft_consensus.cc:2229] T fd80bf2d9e344d989051b5324742d837 P 8c2dc32c75904237a8d40ae792d3c57e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:32:17.166203 13620 raft_consensus.cc:2258] T fd80bf2d9e344d989051b5324742d837 P 8c2dc32c75904237a8d40ae792d3c57e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:32:17.166891 13620 ts_tablet_manager.cc:1905] T fd80bf2d9e344d989051b5324742d837 P 8c2dc32c75904237a8d40ae792d3c57e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:32:17.169924 13620 ts_tablet_manager.cc:1918] T fd80bf2d9e344d989051b5324742d837 P 8c2dc32c75904237a8d40ae792d3c57e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 16:32:17.170014 13620 log.cc:1196] T fd80bf2d9e344d989051b5324742d837 P 8c2dc32c75904237a8d40ae792d3c57e: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-0/wal/wals/fd80bf2d9e344d989051b5324742d837
I20260926 16:32:17.170128 13622 tablet_replica.cc:330] T fd80bf2d9e344d989051b5324742d837 P ebf7b662b0724a89b0f660920b5f4e8e: stopping tablet replica
I20260926 16:32:17.170331 13620 ts_tablet_manager.cc:1939] T fd80bf2d9e344d989051b5324742d837 P 8c2dc32c75904237a8d40ae792d3c57e: Deleting consensus metadata
I20260926 16:32:17.170344 13622 raft_consensus.cc:2229] T fd80bf2d9e344d989051b5324742d837 P ebf7b662b0724a89b0f660920b5f4e8e [term 1 LEADER]: Raft consensus shutting down.
I20260926 16:32:17.170531 13621 tablet_replica.cc:330] T fd80bf2d9e344d989051b5324742d837 P 0003ec5f5cdd45f291cc2505083bcd8e: stopping tablet replica
I20260926 16:32:17.170578 13622 raft_consensus.cc:2258] T fd80bf2d9e344d989051b5324742d837 P ebf7b662b0724a89b0f660920b5f4e8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:32:17.170754 13621 raft_consensus.cc:2229] T fd80bf2d9e344d989051b5324742d837 P 0003ec5f5cdd45f291cc2505083bcd8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:32:17.170940 13621 raft_consensus.cc:2258] T fd80bf2d9e344d989051b5324742d837 P 0003ec5f5cdd45f291cc2505083bcd8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:32:17.171074 13018 catalog_manager.cc:4841] TS 8c2dc32c75904237a8d40ae792d3c57e (127.11.8.193:33355): tablet fd80bf2d9e344d989051b5324742d837 (table test-table [id=56ed8458042f41748e4199c8ee061f0c]) successfully deleted
I20260926 16:32:17.171304 13622 ts_tablet_manager.cc:1905] T fd80bf2d9e344d989051b5324742d837 P ebf7b662b0724a89b0f660920b5f4e8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:32:17.171830 13621 ts_tablet_manager.cc:1905] T fd80bf2d9e344d989051b5324742d837 P 0003ec5f5cdd45f291cc2505083bcd8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:32:17.174299 13622 ts_tablet_manager.cc:1918] T fd80bf2d9e344d989051b5324742d837 P ebf7b662b0724a89b0f660920b5f4e8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 16:32:17.174396 13622 log.cc:1196] T fd80bf2d9e344d989051b5324742d837 P ebf7b662b0724a89b0f660920b5f4e8e: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-1/wal/wals/fd80bf2d9e344d989051b5324742d837
I20260926 16:32:17.174705 13622 ts_tablet_manager.cc:1939] T fd80bf2d9e344d989051b5324742d837 P ebf7b662b0724a89b0f660920b5f4e8e: Deleting consensus metadata
I20260926 16:32:17.174947 13621 ts_tablet_manager.cc:1918] T fd80bf2d9e344d989051b5324742d837 P 0003ec5f5cdd45f291cc2505083bcd8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 16:32:17.175033 13621 log.cc:1196] T fd80bf2d9e344d989051b5324742d837 P 0003ec5f5cdd45f291cc2505083bcd8e: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-2/wal/wals/fd80bf2d9e344d989051b5324742d837
I20260926 16:32:17.175340 13621 ts_tablet_manager.cc:1939] T fd80bf2d9e344d989051b5324742d837 P 0003ec5f5cdd45f291cc2505083bcd8e: Deleting consensus metadata
I20260926 16:32:17.175680 13016 catalog_manager.cc:4841] TS ebf7b662b0724a89b0f660920b5f4e8e (127.11.8.194:43943): tablet fd80bf2d9e344d989051b5324742d837 (table test-table [id=56ed8458042f41748e4199c8ee061f0c]) successfully deleted
I20260926 16:32:17.176146 13019 catalog_manager.cc:4841] TS 0003ec5f5cdd45f291cc2505083bcd8e (127.11.8.195:37479): tablet fd80bf2d9e344d989051b5324742d837 (table test-table [id=56ed8458042f41748e4199c8ee061f0c]) successfully deleted
Sep 26 16:32:17 dist-test-slave-2zvq krb5kdc[12987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440337, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 16:32:17 dist-test-slave-2zvq krb5kdc[12987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440337, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:32:17.197992 13630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:17.183824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45840 (local address 127.11.8.254:34553)
0926 16:32:17.184212 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:17.184219 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:17.184234 (+ 15us) server_negotiation.cc:407] Connection header received
0926 16:32:17.184280 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:17.184283 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:17.184326 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:17.184416 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:17.185085 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:17.185762 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:17.186749 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:17.186999 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:17.187085 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:17.195081 (+ 7996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:17.195084 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:17.195086 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:17.195110 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:17.196457 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:17.196911 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:17.196914 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:17.196916 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:17.196997 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:17.197283 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:17.197285 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:17.197286 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:17.197453 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:17.197569 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:17.197767 (+ 198us) server_negotiation.cc:299] Negotiation successful
0926 16:32:17.197863 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":95,"threads_started":1}
I20260926 16:32:17.200508 13032 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45840:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:32:17.200891 13032 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:32:17.205302 13169 tablet_service.cc:1467] Processing CreateTablet for tablet f1eb64f796344af9aae1519ab5b9b1d5 (DEFAULT_TABLE table=test-table [id=9eff7e19e4cc4944b984514bfffca776]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:17.205381 13309 tablet_service.cc:1467] Processing CreateTablet for tablet f1eb64f796344af9aae1519ab5b9b1d5 (DEFAULT_TABLE table=test-table [id=9eff7e19e4cc4944b984514bfffca776]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:17.205492 13449 tablet_service.cc:1467] Processing CreateTablet for tablet f1eb64f796344af9aae1519ab5b9b1d5 (DEFAULT_TABLE table=test-table [id=9eff7e19e4cc4944b984514bfffca776]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:17.205626 13169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1eb64f796344af9aae1519ab5b9b1d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:17.205685 13309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1eb64f796344af9aae1519ab5b9b1d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:17.205740 13449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1eb64f796344af9aae1519ab5b9b1d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:17.208067 13631 tablet_bootstrap.cc:492] T f1eb64f796344af9aae1519ab5b9b1d5 P 8c2dc32c75904237a8d40ae792d3c57e: Bootstrap starting.
I20260926 16:32:17.208488 13633 tablet_bootstrap.cc:492] T f1eb64f796344af9aae1519ab5b9b1d5 P ebf7b662b0724a89b0f660920b5f4e8e: Bootstrap starting.
I20260926 16:32:17.208493 13632 tablet_bootstrap.cc:492] T f1eb64f796344af9aae1519ab5b9b1d5 P 0003ec5f5cdd45f291cc2505083bcd8e: Bootstrap starting.
I20260926 16:32:17.209244 13633 tablet_bootstrap.cc:654] T f1eb64f796344af9aae1519ab5b9b1d5 P ebf7b662b0724a89b0f660920b5f4e8e: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:17.209412 13632 tablet_bootstrap.cc:654] T f1eb64f796344af9aae1519ab5b9b1d5 P 0003ec5f5cdd45f291cc2505083bcd8e: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:17.209435 13631 tablet_bootstrap.cc:654] T f1eb64f796344af9aae1519ab5b9b1d5 P 8c2dc32c75904237a8d40ae792d3c57e: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:17.210549 13633 tablet_bootstrap.cc:492] T f1eb64f796344af9aae1519ab5b9b1d5 P ebf7b662b0724a89b0f660920b5f4e8e: No bootstrap required, opened a new log
I20260926 16:32:17.210585 13632 tablet_bootstrap.cc:492] T f1eb64f796344af9aae1519ab5b9b1d5 P 0003ec5f5cdd45f291cc2505083bcd8e: No bootstrap required, opened a new log
I20260926 16:32:17.210650 13632 ts_tablet_manager.cc:1397] T f1eb64f796344af9aae1519ab5b9b1d5 P 0003ec5f5cdd45f291cc2505083bcd8e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 16:32:17.210645 13633 ts_tablet_manager.cc:1397] T f1eb64f796344af9aae1519ab5b9b1d5 P ebf7b662b0724a89b0f660920b5f4e8e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 16:32:17.210784 13631 tablet_bootstrap.cc:492] T f1eb64f796344af9aae1519ab5b9b1d5 P 8c2dc32c75904237a8d40ae792d3c57e: No bootstrap required, opened a new log
I20260926 16:32:17.210922 13631 ts_tablet_manager.cc:1397] T f1eb64f796344af9aae1519ab5b9b1d5 P 8c2dc32c75904237a8d40ae792d3c57e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260926 16:32:17.211069 13633 raft_consensus.cc:348] T f1eb64f796344af9aae1519ab5b9b1d5 P ebf7b662b0724a89b0f660920b5f4e8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:17.211069 13632 raft_consensus.cc:348] T f1eb64f796344af9aae1519ab5b9b1d5 P 0003ec5f5cdd45f291cc2505083bcd8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:17.211191 13632 raft_consensus.cc:374] T f1eb64f796344af9aae1519ab5b9b1d5 P 0003ec5f5cdd45f291cc2505083bcd8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:17.211191 13633 raft_consensus.cc:374] T f1eb64f796344af9aae1519ab5b9b1d5 P ebf7b662b0724a89b0f660920b5f4e8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:17.211226 13633 raft_consensus.cc:729] T f1eb64f796344af9aae1519ab5b9b1d5 P ebf7b662b0724a89b0f660920b5f4e8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebf7b662b0724a89b0f660920b5f4e8e, State: Initialized, Role: FOLLOWER
I20260926 16:32:17.211226 13632 raft_consensus.cc:729] T f1eb64f796344af9aae1519ab5b9b1d5 P 0003ec5f5cdd45f291cc2505083bcd8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0003ec5f5cdd45f291cc2505083bcd8e, State: Initialized, Role: FOLLOWER
I20260926 16:32:17.211359 13633 consensus_queue.cc:260] T f1eb64f796344af9aae1519ab5b9b1d5 P ebf7b662b0724a89b0f660920b5f4e8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:17.211380 13632 consensus_queue.cc:260] T f1eb64f796344af9aae1519ab5b9b1d5 P 0003ec5f5cdd45f291cc2505083bcd8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:17.211441 13631 raft_consensus.cc:348] T f1eb64f796344af9aae1519ab5b9b1d5 P 8c2dc32c75904237a8d40ae792d3c57e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:17.211578 13631 raft_consensus.cc:374] T f1eb64f796344af9aae1519ab5b9b1d5 P 8c2dc32c75904237a8d40ae792d3c57e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:17.211611 13631 raft_consensus.cc:729] T f1eb64f796344af9aae1519ab5b9b1d5 P 8c2dc32c75904237a8d40ae792d3c57e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c2dc32c75904237a8d40ae792d3c57e, State: Initialized, Role: FOLLOWER
I20260926 16:32:17.211649 13633 ts_tablet_manager.cc:1428] T f1eb64f796344af9aae1519ab5b9b1d5 P ebf7b662b0724a89b0f660920b5f4e8e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 16:32:17.211709 13632 ts_tablet_manager.cc:1428] T f1eb64f796344af9aae1519ab5b9b1d5 P 0003ec5f5cdd45f291cc2505083bcd8e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 16:32:17.211747 13631 consensus_queue.cc:260] T f1eb64f796344af9aae1519ab5b9b1d5 P 8c2dc32c75904237a8d40ae792d3c57e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:17.212152 13631 ts_tablet_manager.cc:1428] T f1eb64f796344af9aae1519ab5b9b1d5 P 8c2dc32c75904237a8d40ae792d3c57e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 16:32:17.288681 13601 raft_consensus.cc:482] T f1eb64f796344af9aae1519ab5b9b1d5 P 0003ec5f5cdd45f291cc2505083bcd8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:32:17.288800 13601 raft_consensus.cc:504] T f1eb64f796344af9aae1519ab5b9b1d5 P 0003ec5f5cdd45f291cc2505083bcd8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:17.289179 13601 leader_election.cc:288] T f1eb64f796344af9aae1519ab5b9b1d5 P 0003ec5f5cdd45f291cc2505083bcd8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c2dc32c75904237a8d40ae792d3c57e (127.11.8.193:33355), ebf7b662b0724a89b0f660920b5f4e8e (127.11.8.194:43943)
I20260926 16:32:17.289561 13189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1eb64f796344af9aae1519ab5b9b1d5" candidate_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c2dc32c75904237a8d40ae792d3c57e" is_pre_election: true
I20260926 16:32:17.289574 13329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1eb64f796344af9aae1519ab5b9b1d5" candidate_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" is_pre_election: true
I20260926 16:32:17.289710 13329 raft_consensus.cc:2454] T f1eb64f796344af9aae1519ab5b9b1d5 P ebf7b662b0724a89b0f660920b5f4e8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0003ec5f5cdd45f291cc2505083bcd8e in term 0.
I20260926 16:32:17.289710 13189 raft_consensus.cc:2454] T f1eb64f796344af9aae1519ab5b9b1d5 P 8c2dc32c75904237a8d40ae792d3c57e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0003ec5f5cdd45f291cc2505083bcd8e in term 0.
I20260926 16:32:17.290041 13400 leader_election.cc:302] T f1eb64f796344af9aae1519ab5b9b1d5 P 0003ec5f5cdd45f291cc2505083bcd8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0003ec5f5cdd45f291cc2505083bcd8e, ebf7b662b0724a89b0f660920b5f4e8e; no voters:
I20260926 16:32:17.290244 13601 raft_consensus.cc:2781] T f1eb64f796344af9aae1519ab5b9b1d5 P 0003ec5f5cdd45f291cc2505083bcd8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:32:17.290323 13601 raft_consensus.cc:482] T f1eb64f796344af9aae1519ab5b9b1d5 P 0003ec5f5cdd45f291cc2505083bcd8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:32:17.290378 13601 raft_consensus.cc:3037] T f1eb64f796344af9aae1519ab5b9b1d5 P 0003ec5f5cdd45f291cc2505083bcd8e [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:17.291435 13601 raft_consensus.cc:504] T f1eb64f796344af9aae1519ab5b9b1d5 P 0003ec5f5cdd45f291cc2505083bcd8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:17.291774 13601 leader_election.cc:288] T f1eb64f796344af9aae1519ab5b9b1d5 P 0003ec5f5cdd45f291cc2505083bcd8e [CANDIDATE]: Term 1 election: Requested vote from peers 8c2dc32c75904237a8d40ae792d3c57e (127.11.8.193:33355), ebf7b662b0724a89b0f660920b5f4e8e (127.11.8.194:43943)
I20260926 16:32:17.292184 13189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1eb64f796344af9aae1519ab5b9b1d5" candidate_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c2dc32c75904237a8d40ae792d3c57e"
I20260926 16:32:17.292184 13329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1eb64f796344af9aae1519ab5b9b1d5" candidate_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebf7b662b0724a89b0f660920b5f4e8e"
I20260926 16:32:17.292320 13329 raft_consensus.cc:3037] T f1eb64f796344af9aae1519ab5b9b1d5 P ebf7b662b0724a89b0f660920b5f4e8e [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:17.292327 13189 raft_consensus.cc:3037] T f1eb64f796344af9aae1519ab5b9b1d5 P 8c2dc32c75904237a8d40ae792d3c57e [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:17.293555 13329 raft_consensus.cc:2454] T f1eb64f796344af9aae1519ab5b9b1d5 P ebf7b662b0724a89b0f660920b5f4e8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0003ec5f5cdd45f291cc2505083bcd8e in term 1.
I20260926 16:32:17.293555 13189 raft_consensus.cc:2454] T f1eb64f796344af9aae1519ab5b9b1d5 P 8c2dc32c75904237a8d40ae792d3c57e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0003ec5f5cdd45f291cc2505083bcd8e in term 1.
I20260926 16:32:17.293891 13400 leader_election.cc:302] T f1eb64f796344af9aae1519ab5b9b1d5 P 0003ec5f5cdd45f291cc2505083bcd8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0003ec5f5cdd45f291cc2505083bcd8e, 8c2dc32c75904237a8d40ae792d3c57e, ebf7b662b0724a89b0f660920b5f4e8e; no voters:
I20260926 16:32:17.294101 13601 raft_consensus.cc:2781] T f1eb64f796344af9aae1519ab5b9b1d5 P 0003ec5f5cdd45f291cc2505083bcd8e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:32:17.294231 13601 raft_consensus.cc:686] T f1eb64f796344af9aae1519ab5b9b1d5 P 0003ec5f5cdd45f291cc2505083bcd8e [term 1 LEADER]: Becoming Leader. State: Replica: 0003ec5f5cdd45f291cc2505083bcd8e, State: Running, Role: LEADER
I20260926 16:32:17.294396 13601 consensus_queue.cc:237] T f1eb64f796344af9aae1519ab5b9b1d5 P 0003ec5f5cdd45f291cc2505083bcd8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:17.296099 13032 catalog_manager.cc:5495] T f1eb64f796344af9aae1519ab5b9b1d5 P 0003ec5f5cdd45f291cc2505083bcd8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0003ec5f5cdd45f291cc2505083bcd8e (127.11.8.195). New cstate: current_term: 1 leader_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } health_report { overall_health: HEALTHY } } }
I20260926 16:32:17.316641 13640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:17.313324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49102 (local address 127.11.8.195:37479)
0926 16:32:17.313602 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:17.313610 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:17.313630 (+ 20us) server_negotiation.cc:407] Connection header received
0926 16:32:17.313663 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:17.313665 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:17.313705 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:17.313790 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:32:17.314063 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:17.314681 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:17.315550 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:17.315785 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:17.315896 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:17.316173 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:32:17.316252 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:32:17.316335 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:17.316425 (+ 90us) server_negotiation.cc:299] Negotiation successful
0926 16:32:17.316486 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":101,"threads_started":1}
I20260926 16:32:17.324878 13640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:17.322156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:59511 (local address 127.11.8.195:37479)
0926 16:32:17.322260 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:17.322300 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:17.322442 (+ 142us) server_negotiation.cc:407] Connection header received
0926 16:32:17.322525 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:17.322527 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:17.322565 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:17.322625 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:32:17.322839 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:17.323289 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:17.324042 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:17.324234 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:17.324306 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:17.324522 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:32:17.324575 (+ 53us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:32:17.324627 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:17.324708 (+ 81us) server_negotiation.cc:299] Negotiation successful
0926 16:32:17.324763 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":15,"server-negotiator.queue_time_us":46}
I20260926 16:32:17.324893 13643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:17.322105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:37479 (local address 127.11.8.195:59511)
0926 16:32:17.322360 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:17.322372 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:17.322452 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:17.322648 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:17.322650 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:17.322655 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 16:32:17.322764 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:17.322768 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:17.323373 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:17.323375 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:17.323945 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:17.323952 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:17.324329 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:17.324331 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:17.324401 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:17.324425 (+ 24us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 16:32:17.324648 (+ 223us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 16:32:17.324653 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 16:32:17.324702 (+ 49us) client_negotiation.cc:240] Negotiation successful
0926 16:32:17.324779 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":17,"thread_start_us":89,"threads_started":1}
I20260926 16:32:17.330382 13189 raft_consensus.cc:1261] T f1eb64f796344af9aae1519ab5b9b1d5 P 8c2dc32c75904237a8d40ae792d3c57e [term 1 FOLLOWER]: Refusing update from remote peer 0003ec5f5cdd45f291cc2505083bcd8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:32:17.330407 13329 raft_consensus.cc:1261] T f1eb64f796344af9aae1519ab5b9b1d5 P ebf7b662b0724a89b0f660920b5f4e8e [term 1 FOLLOWER]: Refusing update from remote peer 0003ec5f5cdd45f291cc2505083bcd8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:32:17.330752 13601 consensus_queue.cc:1035] T f1eb64f796344af9aae1519ab5b9b1d5 P 0003ec5f5cdd45f291cc2505083bcd8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:32:17.330893 13606 consensus_queue.cc:1035] T f1eb64f796344af9aae1519ab5b9b1d5 P 0003ec5f5cdd45f291cc2505083bcd8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:32:17.363453 13029 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45840:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:32:17.363636 13029 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45840:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:32:17.365700 13029 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d59182d471a4c5596b65f20feab477e: Sending DeleteTablet for 3 replicas of tablet f1eb64f796344af9aae1519ab5b9b1d5
I20260926 16:32:17.366242 13169 tablet_service.cc:1514] Processing DeleteTablet for tablet f1eb64f796344af9aae1519ab5b9b1d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:32:17 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:54260
I20260926 16:32:17.366285 13309 tablet_service.cc:1514] Processing DeleteTablet for tablet f1eb64f796344af9aae1519ab5b9b1d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:32:17 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:55670
I20260926 16:32:17.366448 13622 tablet_replica.cc:330] T f1eb64f796344af9aae1519ab5b9b1d5 P ebf7b662b0724a89b0f660920b5f4e8e: stopping tablet replica
I20260926 16:32:17.366470 13620 tablet_replica.cc:330] T f1eb64f796344af9aae1519ab5b9b1d5 P 8c2dc32c75904237a8d40ae792d3c57e: stopping tablet replica
I20260926 16:32:17.366565 13620 raft_consensus.cc:2229] T f1eb64f796344af9aae1519ab5b9b1d5 P 8c2dc32c75904237a8d40ae792d3c57e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:32:17.366565 13622 raft_consensus.cc:2229] T f1eb64f796344af9aae1519ab5b9b1d5 P ebf7b662b0724a89b0f660920b5f4e8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:32:17.366667 13620 raft_consensus.cc:2258] T f1eb64f796344af9aae1519ab5b9b1d5 P 8c2dc32c75904237a8d40ae792d3c57e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:32:17.366667 13622 raft_consensus.cc:2258] T f1eb64f796344af9aae1519ab5b9b1d5 P ebf7b662b0724a89b0f660920b5f4e8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:32:17.367054 13029 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45840:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 16:32:17.367426 13622 ts_tablet_manager.cc:1905] T f1eb64f796344af9aae1519ab5b9b1d5 P ebf7b662b0724a89b0f660920b5f4e8e: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 16:32:17.367439 13029 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:32:17.367480 13620 ts_tablet_manager.cc:1905] T f1eb64f796344af9aae1519ab5b9b1d5 P 8c2dc32c75904237a8d40ae792d3c57e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:32:17.369491 13448 tablet_service.cc:1514] Processing DeleteTablet for tablet f1eb64f796344af9aae1519ab5b9b1d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:32:17 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:49088
I20260926 16:32:17.369935 13622 ts_tablet_manager.cc:1918] T f1eb64f796344af9aae1519ab5b9b1d5 P ebf7b662b0724a89b0f660920b5f4e8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 16:32:17.369935 13620 ts_tablet_manager.cc:1918] T f1eb64f796344af9aae1519ab5b9b1d5 P 8c2dc32c75904237a8d40ae792d3c57e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 16:32:17.370016 13620 log.cc:1196] T f1eb64f796344af9aae1519ab5b9b1d5 P 8c2dc32c75904237a8d40ae792d3c57e: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-0/wal/wals/f1eb64f796344af9aae1519ab5b9b1d5
I20260926 16:32:17.370016 13622 log.cc:1196] T f1eb64f796344af9aae1519ab5b9b1d5 P ebf7b662b0724a89b0f660920b5f4e8e: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-1/wal/wals/f1eb64f796344af9aae1519ab5b9b1d5
I20260926 16:32:17.370298 13620 ts_tablet_manager.cc:1939] T f1eb64f796344af9aae1519ab5b9b1d5 P 8c2dc32c75904237a8d40ae792d3c57e: Deleting consensus metadata
I20260926 16:32:17.370390 13621 tablet_replica.cc:330] T f1eb64f796344af9aae1519ab5b9b1d5 P 0003ec5f5cdd45f291cc2505083bcd8e: stopping tablet replica
I20260926 16:32:17.370522 13621 raft_consensus.cc:2229] T f1eb64f796344af9aae1519ab5b9b1d5 P 0003ec5f5cdd45f291cc2505083bcd8e [term 1 LEADER]: Raft consensus shutting down.
I20260926 16:32:17.370814 13621 raft_consensus.cc:2258] T f1eb64f796344af9aae1519ab5b9b1d5 P 0003ec5f5cdd45f291cc2505083bcd8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:32:17.370959 13018 catalog_manager.cc:4841] TS 8c2dc32c75904237a8d40ae792d3c57e (127.11.8.193:33355): tablet f1eb64f796344af9aae1519ab5b9b1d5 (table test-table [id=9eff7e19e4cc4944b984514bfffca776]) successfully deleted
I20260926 16:32:17.371546 13621 ts_tablet_manager.cc:1905] T f1eb64f796344af9aae1519ab5b9b1d5 P 0003ec5f5cdd45f291cc2505083bcd8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:32:17.373159 13169 tablet_service.cc:1467] Processing CreateTablet for tablet 60f2b624372c4797b379e3f6867ab14e (DEFAULT_TABLE table=test-table [id=27ede60aa5f549f3a03536ed57e6e1d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:17.373258 13448 tablet_service.cc:1467] Processing CreateTablet for tablet 60f2b624372c4797b379e3f6867ab14e (DEFAULT_TABLE table=test-table [id=27ede60aa5f549f3a03536ed57e6e1d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:17.373425 13169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60f2b624372c4797b379e3f6867ab14e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:17.373523 13448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60f2b624372c4797b379e3f6867ab14e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:17.374151 13621 ts_tablet_manager.cc:1918] T f1eb64f796344af9aae1519ab5b9b1d5 P 0003ec5f5cdd45f291cc2505083bcd8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 16:32:17.374212 13621 log.cc:1196] T f1eb64f796344af9aae1519ab5b9b1d5 P 0003ec5f5cdd45f291cc2505083bcd8e: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-2/wal/wals/f1eb64f796344af9aae1519ab5b9b1d5
I20260926 16:32:17.374516 13621 ts_tablet_manager.cc:1939] T f1eb64f796344af9aae1519ab5b9b1d5 P 0003ec5f5cdd45f291cc2505083bcd8e: Deleting consensus metadata
I20260926 16:32:17.375684 13309 tablet_service.cc:1467] Processing CreateTablet for tablet 60f2b624372c4797b379e3f6867ab14e (DEFAULT_TABLE table=test-table [id=27ede60aa5f549f3a03536ed57e6e1d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:17.375972 13309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60f2b624372c4797b379e3f6867ab14e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:17.376468 13632 tablet_bootstrap.cc:492] T 60f2b624372c4797b379e3f6867ab14e P 0003ec5f5cdd45f291cc2505083bcd8e: Bootstrap starting.
I20260926 16:32:17.377364 13632 tablet_bootstrap.cc:654] T 60f2b624372c4797b379e3f6867ab14e P 0003ec5f5cdd45f291cc2505083bcd8e: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:17.378018 13631 tablet_bootstrap.cc:492] T 60f2b624372c4797b379e3f6867ab14e P 8c2dc32c75904237a8d40ae792d3c57e: Bootstrap starting.
I20260926 16:32:17.378496 13632 tablet_bootstrap.cc:492] T 60f2b624372c4797b379e3f6867ab14e P 0003ec5f5cdd45f291cc2505083bcd8e: No bootstrap required, opened a new log
I20260926 16:32:17.378576 13632 ts_tablet_manager.cc:1397] T 60f2b624372c4797b379e3f6867ab14e P 0003ec5f5cdd45f291cc2505083bcd8e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 16:32:17.378969 13631 tablet_bootstrap.cc:654] T 60f2b624372c4797b379e3f6867ab14e P 8c2dc32c75904237a8d40ae792d3c57e: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:17.378979 13632 raft_consensus.cc:348] T 60f2b624372c4797b379e3f6867ab14e P 0003ec5f5cdd45f291cc2505083bcd8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:17.379096 13632 raft_consensus.cc:374] T 60f2b624372c4797b379e3f6867ab14e P 0003ec5f5cdd45f291cc2505083bcd8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:17.379132 13632 raft_consensus.cc:729] T 60f2b624372c4797b379e3f6867ab14e P 0003ec5f5cdd45f291cc2505083bcd8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0003ec5f5cdd45f291cc2505083bcd8e, State: Initialized, Role: FOLLOWER
I20260926 16:32:17.379252 13632 consensus_queue.cc:260] T 60f2b624372c4797b379e3f6867ab14e P 0003ec5f5cdd45f291cc2505083bcd8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:17.379499 13632 ts_tablet_manager.cc:1428] T 60f2b624372c4797b379e3f6867ab14e P 0003ec5f5cdd45f291cc2505083bcd8e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 16:32:17.379665 13633 tablet_bootstrap.cc:492] T 60f2b624372c4797b379e3f6867ab14e P ebf7b662b0724a89b0f660920b5f4e8e: Bootstrap starting.
I20260926 16:32:17.380281 13019 catalog_manager.cc:4841] TS 0003ec5f5cdd45f291cc2505083bcd8e (127.11.8.195:37479): tablet f1eb64f796344af9aae1519ab5b9b1d5 (table test-table [id=9eff7e19e4cc4944b984514bfffca776]) successfully deleted
I20260926 16:32:17.380520 13633 tablet_bootstrap.cc:654] T 60f2b624372c4797b379e3f6867ab14e P ebf7b662b0724a89b0f660920b5f4e8e: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:17.381613 13633 tablet_bootstrap.cc:492] T 60f2b624372c4797b379e3f6867ab14e P ebf7b662b0724a89b0f660920b5f4e8e: No bootstrap required, opened a new log
I20260926 16:32:17.381718 13633 ts_tablet_manager.cc:1397] T 60f2b624372c4797b379e3f6867ab14e P ebf7b662b0724a89b0f660920b5f4e8e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 16:32:17.382112 13633 raft_consensus.cc:348] T 60f2b624372c4797b379e3f6867ab14e P ebf7b662b0724a89b0f660920b5f4e8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:17.382228 13633 raft_consensus.cc:374] T 60f2b624372c4797b379e3f6867ab14e P ebf7b662b0724a89b0f660920b5f4e8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:17.382309 13633 raft_consensus.cc:729] T 60f2b624372c4797b379e3f6867ab14e P ebf7b662b0724a89b0f660920b5f4e8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebf7b662b0724a89b0f660920b5f4e8e, State: Initialized, Role: FOLLOWER
I20260926 16:32:17.382454 13633 consensus_queue.cc:260] T 60f2b624372c4797b379e3f6867ab14e P ebf7b662b0724a89b0f660920b5f4e8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:17.382740 13633 ts_tablet_manager.cc:1428] T 60f2b624372c4797b379e3f6867ab14e P ebf7b662b0724a89b0f660920b5f4e8e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260926 16:32:17.383270 13631 tablet_bootstrap.cc:492] T 60f2b624372c4797b379e3f6867ab14e P 8c2dc32c75904237a8d40ae792d3c57e: No bootstrap required, opened a new log
I20260926 16:32:17.383388 13631 ts_tablet_manager.cc:1397] T 60f2b624372c4797b379e3f6867ab14e P 8c2dc32c75904237a8d40ae792d3c57e: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260926 16:32:17.383862 13631 raft_consensus.cc:348] T 60f2b624372c4797b379e3f6867ab14e P 8c2dc32c75904237a8d40ae792d3c57e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:17.383993 13631 raft_consensus.cc:374] T 60f2b624372c4797b379e3f6867ab14e P 8c2dc32c75904237a8d40ae792d3c57e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:17.384030 13631 raft_consensus.cc:729] T 60f2b624372c4797b379e3f6867ab14e P 8c2dc32c75904237a8d40ae792d3c57e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c2dc32c75904237a8d40ae792d3c57e, State: Initialized, Role: FOLLOWER
I20260926 16:32:17.384155 13631 consensus_queue.cc:260] T 60f2b624372c4797b379e3f6867ab14e P 8c2dc32c75904237a8d40ae792d3c57e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:17.384509 13631 ts_tablet_manager.cc:1428] T 60f2b624372c4797b379e3f6867ab14e P 8c2dc32c75904237a8d40ae792d3c57e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 16:32:17.384647 13622 ts_tablet_manager.cc:1939] T f1eb64f796344af9aae1519ab5b9b1d5 P ebf7b662b0724a89b0f660920b5f4e8e: Deleting consensus metadata
I20260926 16:32:17.385265 13016 catalog_manager.cc:4841] TS ebf7b662b0724a89b0f660920b5f4e8e (127.11.8.194:43943): tablet f1eb64f796344af9aae1519ab5b9b1d5 (table test-table [id=9eff7e19e4cc4944b984514bfffca776]) successfully deleted
I20260926 16:32:17.586999 13637 raft_consensus.cc:482] T 60f2b624372c4797b379e3f6867ab14e P 8c2dc32c75904237a8d40ae792d3c57e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:32:17.587182 13637 raft_consensus.cc:504] T 60f2b624372c4797b379e3f6867ab14e P 8c2dc32c75904237a8d40ae792d3c57e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:17.588361 13637 leader_election.cc:288] T 60f2b624372c4797b379e3f6867ab14e P 8c2dc32c75904237a8d40ae792d3c57e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ebf7b662b0724a89b0f660920b5f4e8e (127.11.8.194:43943), 0003ec5f5cdd45f291cc2505083bcd8e (127.11.8.195:37479)
I20260926 16:32:17.593080 13651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:17.588712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:33433 (local address 127.11.8.194:43943)
0926 16:32:17.588989 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:17.589013 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:17.589028 (+ 15us) server_negotiation.cc:407] Connection header received
0926 16:32:17.589212 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:17.589215 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:17.589292 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:17.589390 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:17.589817 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:17.590662 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:17.592070 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:17.592497 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:17.592625 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:17.592682 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:17.592871 (+ 189us) server_negotiation.cc:299] Negotiation successful
0926 16:32:17.592968 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":90,"threads_started":1}
I20260926 16:32:17.593086 13650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:17.588590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:43943 (local address 127.11.8.193:33433)
0926 16:32:17.588928 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:17.588943 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:17.589070 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:17.589415 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:17.589418 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:17.589450 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:17.589672 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:17.589677 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:17.590812 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:17.590815 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:17.591906 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:17.591919 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:17.592618 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:17.592622 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:17.592724 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:17.592746 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 16:32:17.592839 (+ 93us) client_negotiation.cc:240] Negotiation successful
0926 16:32:17.592913 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":131,"spinlock_wait_cycles":12416,"thread_start_us":105,"threads_started":1}
I20260926 16:32:17.593171 13640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:17.588768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:46795 (local address 127.11.8.195:37479)
0926 16:32:17.588902 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:17.588907 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:17.589168 (+ 261us) server_negotiation.cc:407] Connection header received
0926 16:32:17.589258 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:17.589261 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:17.589305 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:17.589390 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:17.589817 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:17.590652 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:17.592077 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:17.592567 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:17.592641 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:17.592681 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:17.592960 (+ 279us) server_negotiation.cc:299] Negotiation successful
0926 16:32:17.593042 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 16:32:17.593606 13652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:17.588683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:37479 (local address 127.11.8.193:46795)
0926 16:32:17.589070 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:17.589083 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:17.589167 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:17.589415 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:17.589418 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:17.589443 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:17.589667 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:17.589675 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:17.590843 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:17.590846 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:17.591916 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:17.591927 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:17.592774 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:17.592777 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:17.592847 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:17.592862 (+ 15us) client_negotiation.cc:769] Sending connection context
0926 16:32:17.592975 (+ 113us) client_negotiation.cc:240] Negotiation successful
0926 16:32:17.593041 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":133,"spinlock_wait_cycles":1920,"thread_start_us":108,"threads_started":1}
I20260926 16:32:17.593792 13329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60f2b624372c4797b379e3f6867ab14e" candidate_uuid: "8c2dc32c75904237a8d40ae792d3c57e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" is_pre_election: true
I20260926 16:32:17.593966 13329 raft_consensus.cc:2454] T 60f2b624372c4797b379e3f6867ab14e P ebf7b662b0724a89b0f660920b5f4e8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c2dc32c75904237a8d40ae792d3c57e in term 0.
I20260926 16:32:17.594122 13469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60f2b624372c4797b379e3f6867ab14e" candidate_uuid: "8c2dc32c75904237a8d40ae792d3c57e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" is_pre_election: true
I20260926 16:32:17.594306 13469 raft_consensus.cc:2454] T 60f2b624372c4797b379e3f6867ab14e P 0003ec5f5cdd45f291cc2505083bcd8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c2dc32c75904237a8d40ae792d3c57e in term 0.
I20260926 16:32:17.594377 13120 leader_election.cc:302] T 60f2b624372c4797b379e3f6867ab14e P 8c2dc32c75904237a8d40ae792d3c57e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c2dc32c75904237a8d40ae792d3c57e, ebf7b662b0724a89b0f660920b5f4e8e; no voters:
I20260926 16:32:17.594669 13637 raft_consensus.cc:2781] T 60f2b624372c4797b379e3f6867ab14e P 8c2dc32c75904237a8d40ae792d3c57e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:32:17.594744 13637 raft_consensus.cc:482] T 60f2b624372c4797b379e3f6867ab14e P 8c2dc32c75904237a8d40ae792d3c57e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:32:17.594794 13637 raft_consensus.cc:3037] T 60f2b624372c4797b379e3f6867ab14e P 8c2dc32c75904237a8d40ae792d3c57e [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:17.595793 13637 raft_consensus.cc:504] T 60f2b624372c4797b379e3f6867ab14e P 8c2dc32c75904237a8d40ae792d3c57e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:17.596158 13637 leader_election.cc:288] T 60f2b624372c4797b379e3f6867ab14e P 8c2dc32c75904237a8d40ae792d3c57e [CANDIDATE]: Term 1 election: Requested vote from peers ebf7b662b0724a89b0f660920b5f4e8e (127.11.8.194:43943), 0003ec5f5cdd45f291cc2505083bcd8e (127.11.8.195:37479)
I20260926 16:32:17.596544 13329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60f2b624372c4797b379e3f6867ab14e" candidate_uuid: "8c2dc32c75904237a8d40ae792d3c57e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebf7b662b0724a89b0f660920b5f4e8e"
I20260926 16:32:17.596544 13469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60f2b624372c4797b379e3f6867ab14e" candidate_uuid: "8c2dc32c75904237a8d40ae792d3c57e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0003ec5f5cdd45f291cc2505083bcd8e"
I20260926 16:32:17.596670 13329 raft_consensus.cc:3037] T 60f2b624372c4797b379e3f6867ab14e P ebf7b662b0724a89b0f660920b5f4e8e [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:17.596670 13469 raft_consensus.cc:3037] T 60f2b624372c4797b379e3f6867ab14e P 0003ec5f5cdd45f291cc2505083bcd8e [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:17.597852 13329 raft_consensus.cc:2454] T 60f2b624372c4797b379e3f6867ab14e P ebf7b662b0724a89b0f660920b5f4e8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c2dc32c75904237a8d40ae792d3c57e in term 1.
I20260926 16:32:17.597857 13469 raft_consensus.cc:2454] T 60f2b624372c4797b379e3f6867ab14e P 0003ec5f5cdd45f291cc2505083bcd8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c2dc32c75904237a8d40ae792d3c57e in term 1.
I20260926 16:32:17.598162 13123 leader_election.cc:302] T 60f2b624372c4797b379e3f6867ab14e P 8c2dc32c75904237a8d40ae792d3c57e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0003ec5f5cdd45f291cc2505083bcd8e, 8c2dc32c75904237a8d40ae792d3c57e; no voters:
I20260926 16:32:17.598409 13637 raft_consensus.cc:2781] T 60f2b624372c4797b379e3f6867ab14e P 8c2dc32c75904237a8d40ae792d3c57e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:32:17.598625 13637 raft_consensus.cc:686] T 60f2b624372c4797b379e3f6867ab14e P 8c2dc32c75904237a8d40ae792d3c57e [term 1 LEADER]: Becoming Leader. State: Replica: 8c2dc32c75904237a8d40ae792d3c57e, State: Running, Role: LEADER
I20260926 16:32:17.598886 13637 consensus_queue.cc:237] T 60f2b624372c4797b379e3f6867ab14e P 8c2dc32c75904237a8d40ae792d3c57e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } }
I20260926 16:32:17.601091 13032 catalog_manager.cc:5495] T 60f2b624372c4797b379e3f6867ab14e P 8c2dc32c75904237a8d40ae792d3c57e reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c2dc32c75904237a8d40ae792d3c57e (127.11.8.193). New cstate: current_term: 1 leader_uuid: "8c2dc32c75904237a8d40ae792d3c57e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c2dc32c75904237a8d40ae792d3c57e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33355 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 } health_report { overall_health: UNKNOWN } } }
I20260926 16:32:17.627058 13655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:17.623646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33554 (local address 127.11.8.193:33355)
0926 16:32:17.623916 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:17.623924 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:17.623936 (+ 12us) server_negotiation.cc:407] Connection header received
0926 16:32:17.624007 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:17.624009 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:17.624061 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:17.624165 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:32:17.624342 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:17.624862 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:17.625654 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:17.625886 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:17.625971 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:17.626243 (+ 272us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:32:17.626726 (+ 483us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:32:17.626817 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:17.626887 (+ 70us) server_negotiation.cc:299] Negotiation successful
0926 16:32:17.626930 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":120,"threads_started":1}
I20260926 16:32:17.637120 13640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:17.634129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:50295 (local address 127.11.8.195:37479)
0926 16:32:17.634257 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:17.634299 (+ 42us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:17.634319 (+ 20us) server_negotiation.cc:407] Connection header received
0926 16:32:17.634461 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:17.634464 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:17.634513 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:17.634575 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:32:17.634786 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:17.635257 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:17.636081 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:17.636299 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:17.636388 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:17.636715 (+ 327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:32:17.636797 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:32:17.636881 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:17.636979 (+ 98us) server_negotiation.cc:299] Negotiation successful
0926 16:32:17.637024 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 16:32:17.637143 13658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:17.633912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:37479 (local address 127.11.8.193:50295)
0926 16:32:17.634248 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:17.634260 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:17.634353 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:17.634597 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:17.634599 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:17.634605 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 16:32:17.634695 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:17.634699 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:17.635337 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:17.635339 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:17.635937 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:17.635945 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:17.636418 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:17.636420 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:17.636504 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:17.636563 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 16:32:17.636890 (+ 327us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 16:32:17.636895 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 16:32:17.636953 (+ 58us) client_negotiation.cc:240] Negotiation successful
0926 16:32:17.637014 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":88,"threads_started":1}
I20260926 16:32:17.642872 13329 raft_consensus.cc:1261] T 60f2b624372c4797b379e3f6867ab14e P ebf7b662b0724a89b0f660920b5f4e8e [term 1 FOLLOWER]: Refusing update from remote peer 8c2dc32c75904237a8d40ae792d3c57e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:32:17.642880 13469 raft_consensus.cc:1261] T 60f2b624372c4797b379e3f6867ab14e P 0003ec5f5cdd45f291cc2505083bcd8e [term 1 FOLLOWER]: Refusing update from remote peer 8c2dc32c75904237a8d40ae792d3c57e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:32:17.643353 13653 consensus_queue.cc:1035] T 60f2b624372c4797b379e3f6867ab14e P 8c2dc32c75904237a8d40ae792d3c57e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0003ec5f5cdd45f291cc2505083bcd8e" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:32:17.643604 13637 consensus_queue.cc:1035] T 60f2b624372c4797b379e3f6867ab14e P 8c2dc32c75904237a8d40ae792d3c57e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebf7b662b0724a89b0f660920b5f4e8e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:32:17.660070 13643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:17.656612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:33355 (local address 127.11.8.195:37529)
0926 16:32:17.656725 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:17.656741 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:17.656820 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:17.657076 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:17.657079 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:17.657086 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 16:32:17.657190 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:17.657215 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:17.658006 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:17.658009 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:17.658842 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:17.658854 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:17.659378 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:17.659384 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:17.659476 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:17.659506 (+ 30us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 16:32:17.659808 (+ 302us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 16:32:17.659816 (+ 8us) client_negotiation.cc:769] Sending connection context
0926 16:32:17.659880 (+ 64us) client_negotiation.cc:240] Negotiation successful
0926 16:32:17.659960 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260926 16:32:17.660070 13655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:17.656652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:37529 (local address 127.11.8.193:33355)
0926 16:32:17.656767 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:17.656772 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:17.656789 (+ 17us) server_negotiation.cc:407] Connection header received
0926 16:32:17.656907 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:17.656912 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:17.656974 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:17.657053 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:32:17.657320 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:17.657854 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:17.658975 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:17.659293 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:17.659410 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:17.659619 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:32:17.659703 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:32:17.659797 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:17.659889 (+ 92us) server_negotiation.cc:299] Negotiation successful
0926 16:32:17.659939 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260926 16:32:17.660746 13603 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260926 16:32:19.224588 13029 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45840:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:32:19.224777 13029 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45840:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:32:19.227321 13029 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d59182d471a4c5596b65f20feab477e: Sending DeleteTablet for 3 replicas of tablet 60f2b624372c4797b379e3f6867ab14e
I20260926 16:32:19.227943 13169 tablet_service.cc:1514] Processing DeleteTablet for tablet 60f2b624372c4797b379e3f6867ab14e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:32:19 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:54260
I20260926 16:32:19.227977 13309 tablet_service.cc:1514] Processing DeleteTablet for tablet 60f2b624372c4797b379e3f6867ab14e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:32:19 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:55670
I20260926 16:32:19.228116 13448 tablet_service.cc:1514] Processing DeleteTablet for tablet 60f2b624372c4797b379e3f6867ab14e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:32:19 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:49088
I20260926 16:32:19.228377 13685 tablet_replica.cc:330] T 60f2b624372c4797b379e3f6867ab14e P 8c2dc32c75904237a8d40ae792d3c57e: stopping tablet replica
I20260926 16:32:19.228420 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 13104
I20260926 16:32:19.228474 13686 tablet_replica.cc:330] T 60f2b624372c4797b379e3f6867ab14e P ebf7b662b0724a89b0f660920b5f4e8e: stopping tablet replica
I20260926 16:32:19.229204 13687 tablet_replica.cc:330] T 60f2b624372c4797b379e3f6867ab14e P 0003ec5f5cdd45f291cc2505083bcd8e: stopping tablet replica
I20260926 16:32:19.229226 13686 raft_consensus.cc:2229] T 60f2b624372c4797b379e3f6867ab14e P ebf7b662b0724a89b0f660920b5f4e8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:32:19.229319 13687 raft_consensus.cc:2229] T 60f2b624372c4797b379e3f6867ab14e P 0003ec5f5cdd45f291cc2505083bcd8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:32:19.229364 13686 raft_consensus.cc:2258] T 60f2b624372c4797b379e3f6867ab14e P ebf7b662b0724a89b0f660920b5f4e8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:32:19.229434 13687 raft_consensus.cc:2258] T 60f2b624372c4797b379e3f6867ab14e P 0003ec5f5cdd45f291cc2505083bcd8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:32:19.230146 13686 ts_tablet_manager.cc:1905] T 60f2b624372c4797b379e3f6867ab14e P ebf7b662b0724a89b0f660920b5f4e8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:32:19.231739 13687 ts_tablet_manager.cc:1905] T 60f2b624372c4797b379e3f6867ab14e P 0003ec5f5cdd45f291cc2505083bcd8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:32:19.232802 13686 ts_tablet_manager.cc:1918] T 60f2b624372c4797b379e3f6867ab14e P ebf7b662b0724a89b0f660920b5f4e8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 16:32:19.232954 13686 log.cc:1196] T 60f2b624372c4797b379e3f6867ab14e P ebf7b662b0724a89b0f660920b5f4e8e: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-1/wal/wals/60f2b624372c4797b379e3f6867ab14e
I20260926 16:32:19.233233 13686 ts_tablet_manager.cc:1939] T 60f2b624372c4797b379e3f6867ab14e P ebf7b662b0724a89b0f660920b5f4e8e: Deleting consensus metadata
I20260926 16:32:19.233925 13016 catalog_manager.cc:4841] TS ebf7b662b0724a89b0f660920b5f4e8e (127.11.8.194:43943): tablet 60f2b624372c4797b379e3f6867ab14e (table test-table [id=27ede60aa5f549f3a03536ed57e6e1d4]) successfully deleted
I20260926 16:32:19.235148 13687 ts_tablet_manager.cc:1918] T 60f2b624372c4797b379e3f6867ab14e P 0003ec5f5cdd45f291cc2505083bcd8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 16:32:19.235245 13687 log.cc:1196] T 60f2b624372c4797b379e3f6867ab14e P 0003ec5f5cdd45f291cc2505083bcd8e: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790440324823152-11299-0/minicluster-data/ts-2/wal/wals/60f2b624372c4797b379e3f6867ab14e
I20260926 16:32:19.235507 13687 ts_tablet_manager.cc:1939] T 60f2b624372c4797b379e3f6867ab14e P 0003ec5f5cdd45f291cc2505083bcd8e: Deleting consensus metadata
I20260926 16:32:19.236291 13019 catalog_manager.cc:4841] TS 0003ec5f5cdd45f291cc2505083bcd8e (127.11.8.195:37479): tablet 60f2b624372c4797b379e3f6867ab14e (table test-table [id=27ede60aa5f549f3a03536ed57e6e1d4]) successfully deleted
W20260926 16:32:19.236841 13018 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.8.193:33355 (error 108)
I20260926 16:32:19.237051 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 13244
I20260926 16:32:19.238444 13689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:19.238009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:33355 (local address 127.0.0.1:33570)
0926 16:32:19.238229 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:19.238348 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.193:33355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":92,"threads_started":1}
W20260926 16:32:19.238647 13018 catalog_manager.cc:4584] TS 8c2dc32c75904237a8d40ae792d3c57e (127.11.8.193:33355): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 60f2b624372c4797b379e3f6867ab14e: Network error: Client connection negotiation failed: client connection to 127.11.8.193:33355: connect: Connection refused (error 111)
I20260926 16:32:19.244287 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 13384
I20260926 16:32:19.251132 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 13000
2026-09-26T16:32:19Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6597 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:32:19 dist-test-slave-2zvq krb5kdc[13692](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:32:19 dist-test-slave-2zvq krb5kdc[13692](info): set up 2 sockets
Sep 26 16:32:19 dist-test-slave-2zvq krb5kdc[13692](info): commencing operation
krb5kdc: starting...
W20260926 16:32:20.296022 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:32:20 dist-test-slave-2zvq krb5kdc[13692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:32:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:32:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
I20260926 16:32:20.413220 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:39733
--webserver_interface=127.11.8.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:43891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:39733
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:20.521754 13705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:20.521991 13705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:20.522029 13705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:20.525926 13705 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:32:20.525977 13705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:20.525996 13705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:20.526012 13705 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:32:20.526026 13705 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:32:20.531036 13705 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:43891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:39733
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:39733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:20.531915 13705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:20.532832 13705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:20.541586 13710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:20.541651 13705 server_base.cc:1034] running on GCE node
W20260926 16:32:20.541586 13711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:20.541586 13713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:20.542461 13705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:20.543540 13705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:20.544695 13705 hybrid_clock.cc:648] HybridClock initialized: now 1790440340544680 us; error 27 us; skew 500 ppm
Sep 26 16:32:20 dist-test-slave-2zvq krb5kdc[13692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440340, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:20.547412 13705 init.cc:377] Logged in from keytab as kudu/127.11.8.254@KRBTEST.COM (short username kudu)
I20260926 16:32:20.548187 13705 webserver.cc:460] Webserver started at http://127.11.8.254:36269/ using document root <none> and password file <none>
I20260926 16:32:20.548738 13705 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:20.548799 13705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:20.548956 13705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:20.550957 13705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "3940b81cb1db4fd68333597c1024d877"
format_stamp: "Formatted at 2026-09-26 16:32:20 on dist-test-slave-2zvq"
server_key: "0b8b78fb07938fe1ccf3a95c236f4be9"
server_key_iv: "c513db9dcc2d4cfa27eb296a7826c359"
server_key_version: "encryptionkey@0"
I20260926 16:32:20.551465 13705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "3940b81cb1db4fd68333597c1024d877"
format_stamp: "Formatted at 2026-09-26 16:32:20 on dist-test-slave-2zvq"
server_key: "0b8b78fb07938fe1ccf3a95c236f4be9"
server_key_iv: "c513db9dcc2d4cfa27eb296a7826c359"
server_key_version: "encryptionkey@0"
I20260926 16:32:20.554935 13705 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 16:32:20.557062 13720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:20.558231 13705 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:32:20.558426 13705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "3940b81cb1db4fd68333597c1024d877"
format_stamp: "Formatted at 2026-09-26 16:32:20 on dist-test-slave-2zvq"
server_key: "0b8b78fb07938fe1ccf3a95c236f4be9"
server_key_iv: "c513db9dcc2d4cfa27eb296a7826c359"
server_key_version: "encryptionkey@0"
I20260926 16:32:20.558548 13705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:20.572372 13705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:20.575205 13705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:20.575397 13705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:20.582334 13705 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:39733
I20260926 16:32:20.582363 13772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:39733 every 8 connection(s)
I20260926 16:32:20.583299 13705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:32:20.586084 13773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:20.589005 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 13705
I20260926 16:32:20.589088 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:32:20.589355 11299 external_mini_cluster.cc:1428] Setting key 21a152d12db9a5cbe6d98376094561c3
I20260926 16:32:20.592209 13773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3940b81cb1db4fd68333597c1024d877: Bootstrap starting.
I20260926 16:32:20.594635 13773 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3940b81cb1db4fd68333597c1024d877: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:20.595392 13773 log.cc:824] T 00000000000000000000000000000000 P 3940b81cb1db4fd68333597c1024d877: Log is configured to *not* fsync() on all Append() calls
I20260926 16:32:20.597745 13773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3940b81cb1db4fd68333597c1024d877: No bootstrap required, opened a new log
Sep 26 16:32:20 dist-test-slave-2zvq krb5kdc[13692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:32:20.600765 13773 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3940b81cb1db4fd68333597c1024d877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3940b81cb1db4fd68333597c1024d877" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 39733 } }
I20260926 16:32:20.601008 13773 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3940b81cb1db4fd68333597c1024d877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:20.601089 13773 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3940b81cb1db4fd68333597c1024d877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3940b81cb1db4fd68333597c1024d877, State: Initialized, Role: FOLLOWER
I20260926 16:32:20.601552 13773 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3940b81cb1db4fd68333597c1024d877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3940b81cb1db4fd68333597c1024d877" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 39733 } }
I20260926 16:32:20.601708 13773 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3940b81cb1db4fd68333597c1024d877 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:32:20.601790 13773 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3940b81cb1db4fd68333597c1024d877 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:32:20.601922 13773 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3940b81cb1db4fd68333597c1024d877 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:20.602876 13773 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3940b81cb1db4fd68333597c1024d877 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3940b81cb1db4fd68333597c1024d877" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 39733 } }
I20260926 16:32:20.603224 13773 leader_election.cc:302] T 00000000000000000000000000000000 P 3940b81cb1db4fd68333597c1024d877 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3940b81cb1db4fd68333597c1024d877; no voters:
I20260926 16:32:20.603617 13773 leader_election.cc:288] T 00000000000000000000000000000000 P 3940b81cb1db4fd68333597c1024d877 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:32:20.603711 13778 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3940b81cb1db4fd68333597c1024d877 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:32:20.603941 13778 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3940b81cb1db4fd68333597c1024d877 [term 1 LEADER]: Becoming Leader. State: Replica: 3940b81cb1db4fd68333597c1024d877, State: Running, Role: LEADER
I20260926 16:32:20.604281 13778 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3940b81cb1db4fd68333597c1024d877 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3940b81cb1db4fd68333597c1024d877" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 39733 } }
I20260926 16:32:20.604501 13776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:20.590755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52220 (local address 127.11.8.254:39733)
0926 16:32:20.591205 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:20.591219 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:20.591254 (+ 35us) server_negotiation.cc:407] Connection header received
0926 16:32:20.591860 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:20.591877 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:20.592204 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:20.592604 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:20.593443 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:20.594484 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:20.595397 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:20.596039 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:20.596207 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:20.598994 (+ 2787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:20.598999 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:20.599011 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:20.599036 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:20.601317 (+ 2281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:20.601757 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:20.601762 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:20.601768 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:20.601926 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:20.602200 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:20.602203 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:20.602205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:20.602685 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:20.602890 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:20.603168 (+ 278us) server_negotiation.cc:299] Negotiation successful
0926 16:32:20.603476 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":149,"threads_started":1}
I20260926 16:32:20.604990 13773 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3940b81cb1db4fd68333597c1024d877 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:32:20.606302 13780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3940b81cb1db4fd68333597c1024d877 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3940b81cb1db4fd68333597c1024d877. Latest consensus state: current_term: 1 leader_uuid: "3940b81cb1db4fd68333597c1024d877" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3940b81cb1db4fd68333597c1024d877" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 39733 } } }
I20260926 16:32:20.606426 13780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3940b81cb1db4fd68333597c1024d877 [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:20.606849 13779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3940b81cb1db4fd68333597c1024d877 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3940b81cb1db4fd68333597c1024d877" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3940b81cb1db4fd68333597c1024d877" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 39733 } } }
I20260926 16:32:20.607004 13779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3940b81cb1db4fd68333597c1024d877 [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:20.607506 13788 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:32:20.610154 13788 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:32:20.615275 13788 catalog_manager.cc:1347] Generated new cluster ID: 7204497fc72845649e556a103cc4038d
I20260926 16:32:20.615337 13788 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:32:20.632843 13788 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:32:20.633872 13788 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:32:20.640955 13788 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3940b81cb1db4fd68333597c1024d877: Generated new TSK 0
I20260926 16:32:20.641630 13788 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.8.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab.
Entry for principal kudu/127.11.8.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab.
I20260926 16:32:20.667598 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.193:0
--local_ip_for_outbound_sockets=127.11.8.193
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:39733
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:43891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:20.780613 13799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:20.780929 13799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:20.781020 13799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:20.785069 13799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:20.785151 13799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:20.785243 13799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.193
I20260926 16:32:20.790649 13799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:43891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:39733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.193
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:20.791654 13799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:20.792665 13799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:20.801947 13805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:20.801888 13804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:20.801888 13807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:20.802309 13799 server_base.cc:1034] running on GCE node
I20260926 16:32:20.802680 13799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:20.803227 13799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:20.804430 13799 hybrid_clock.cc:648] HybridClock initialized: now 1790440340804407 us; error 29 us; skew 500 ppm
Sep 26 16:32:20 dist-test-slave-2zvq krb5kdc[13692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440340, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:20.807471 13799 init.cc:377] Logged in from keytab as kudu/127.11.8.193@KRBTEST.COM (short username kudu)
I20260926 16:32:20.808358 13799 webserver.cc:460] Webserver started at http://127.11.8.193:44825/ using document root <none> and password file <none>
I20260926 16:32:20.808917 13799 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:20.809010 13799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:20.809206 13799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:20.811230 13799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-0/data/instance:
uuid: "f587c78631cd420391c1c71efc1c6bc2"
format_stamp: "Formatted at 2026-09-26 16:32:20 on dist-test-slave-2zvq"
server_key: "752fd4fe5461494b2e331cf12cc7906a"
server_key_iv: "a48f2c5b3e473c288731c0418009a3e2"
server_key_version: "encryptionkey@0"
I20260926 16:32:20.811795 13799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance:
uuid: "f587c78631cd420391c1c71efc1c6bc2"
format_stamp: "Formatted at 2026-09-26 16:32:20 on dist-test-slave-2zvq"
server_key: "752fd4fe5461494b2e331cf12cc7906a"
server_key_iv: "a48f2c5b3e473c288731c0418009a3e2"
server_key_version: "encryptionkey@0"
I20260926 16:32:20.815382 13799 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 16:32:20.817847 13814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:20.819242 13799 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:32:20.819375 13799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-0/wal
uuid: "f587c78631cd420391c1c71efc1c6bc2"
format_stamp: "Formatted at 2026-09-26 16:32:20 on dist-test-slave-2zvq"
server_key: "752fd4fe5461494b2e331cf12cc7906a"
server_key_iv: "a48f2c5b3e473c288731c0418009a3e2"
server_key_version: "encryptionkey@0"
I20260926 16:32:20.819499 13799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:20.834475 13799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:20.837708 13799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:20.837913 13799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:20.838521 13799 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:32:20.839325 13799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:20.839370 13799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:20.839443 13799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:20.839474 13799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:20.847975 13799 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.193:36979
I20260926 16:32:20.848011 13927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.193:36979 every 8 connection(s)
I20260926 16:32:20.848801 13799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
I20260926 16:32:20.853961 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 13799
I20260926 16:32:20.854050 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance
I20260926 16:32:20.854377 11299 external_mini_cluster.cc:1428] Setting key 5f05fed47e4b6361041936db06edba40
Sep 26 16:32:20 dist-test-slave-2zvq krb5kdc[13692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440340, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:32:20.863379 13776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:20.850566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:49361 (local address 127.11.8.254:39733)
0926 16:32:20.850716 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:20.850721 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:20.851983 (+ 1262us) server_negotiation.cc:407] Connection header received
0926 16:32:20.852952 (+ 969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:20.852956 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:20.853001 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:20.853075 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:20.854551 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:20.855155 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:20.856301 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:20.856543 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:20.856655 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:20.859499 (+ 2844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:20.859502 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:20.859504 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:20.859527 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:20.860991 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:20.861653 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:20.861655 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:20.861656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:20.861760 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:20.862154 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:20.862157 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:20.862158 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:20.862404 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:20.862489 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:20.863152 (+ 663us) server_negotiation.cc:299] Negotiation successful
0926 16:32:20.863259 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
WARNING: no policy specified for kudu/127.11.8.194@KRBTEST.COM; defaulting to no policy
I20260926 16:32:20.864717 13930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:20.850963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:39733 (local address 127.11.8.193:49361)
0926 16:32:20.851836 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:20.851877 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:20.852729 (+ 852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:20.853267 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:20.853276 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:20.853755 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:20.854394 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:20.854428 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:20.855289 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:20.855293 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:20.856172 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:20.856183 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:20.856695 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:20.856699 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:20.856870 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:20.857466 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:20.857482 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:20.859317 (+ 1835us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:20.861124 (+ 1807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:20.861133 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:20.861152 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:20.861554 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:20.861858 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:20.861862 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:20.861864 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:20.862058 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:20.862513 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:20.862524 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:20.862892 (+ 368us) client_negotiation.cc:769] Sending connection context
0926 16:32:20.863126 (+ 234us) client_negotiation.cc:240] Negotiation successful
0926 16:32:20.863516 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":663,"thread_start_us":524,"threads_started":1}
I20260926 16:32:20.866029 13928 heartbeater.cc:347] Connected to a master server at 127.11.8.254:39733
I20260926 16:32:20.866322 13928 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:20.866811 13928 heartbeater.cc:511] Master 127.11.8.254:39733 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.194@KRBTEST.COM" created.
I20260926 16:32:20.868391 13737 ts_manager.cc:194] Registered new tserver with Master: f587c78631cd420391c1c71efc1c6bc2 (127.11.8.193:36979)
I20260926 16:32:20.869506 13737 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.193@KRBTEST.COM'} at 127.11.8.193:49361
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab.
Entry for principal kudu/127.11.8.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab.
I20260926 16:32:20.881968 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.194:0
--local_ip_for_outbound_sockets=127.11.8.194
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:39733
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:43891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:20.990613 13933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:20.990870 13933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:20.990912 13933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:20.994995 13933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:20.995045 13933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:20.995133 13933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.194
I20260926 16:32:21.001703 13933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:43891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:39733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.194
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:21.002743 13933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:21.003732 13933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:21.013453 13941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:21.013540 13933 server_base.cc:1034] running on GCE node
W20260926 16:32:21.013463 13939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:21.013478 13938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:21.014161 13933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:21.014856 13933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:21.016062 13933 hybrid_clock.cc:648] HybridClock initialized: now 1790440341016033 us; error 39 us; skew 500 ppm
Sep 26 16:32:21 dist-test-slave-2zvq krb5kdc[13692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440341, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:21.019136 13933 init.cc:377] Logged in from keytab as kudu/127.11.8.194@KRBTEST.COM (short username kudu)
I20260926 16:32:21.019999 13933 webserver.cc:460] Webserver started at http://127.11.8.194:38867/ using document root <none> and password file <none>
I20260926 16:32:21.020534 13933 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:21.020592 13933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:21.020756 13933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:21.022866 13933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-1/data/instance:
uuid: "50c1ffe9e8184a70b25e3657c3438146"
format_stamp: "Formatted at 2026-09-26 16:32:21 on dist-test-slave-2zvq"
server_key: "8783cc5df1c7e39473cf353a81f5337a"
server_key_iv: "2e7f52079d113f37e4f524e2799ab405"
server_key_version: "encryptionkey@0"
I20260926 16:32:21.023463 13933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance:
uuid: "50c1ffe9e8184a70b25e3657c3438146"
format_stamp: "Formatted at 2026-09-26 16:32:21 on dist-test-slave-2zvq"
server_key: "8783cc5df1c7e39473cf353a81f5337a"
server_key_iv: "2e7f52079d113f37e4f524e2799ab405"
server_key_version: "encryptionkey@0"
I20260926 16:32:21.027240 13933 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260926 16:32:21.029574 13948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:21.030836 13933 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260926 16:32:21.030956 13933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-1/wal
uuid: "50c1ffe9e8184a70b25e3657c3438146"
format_stamp: "Formatted at 2026-09-26 16:32:21 on dist-test-slave-2zvq"
server_key: "8783cc5df1c7e39473cf353a81f5337a"
server_key_iv: "2e7f52079d113f37e4f524e2799ab405"
server_key_version: "encryptionkey@0"
I20260926 16:32:21.031057 13933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:21.044024 13933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:21.047240 13933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:21.047451 13933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:21.048022 13933 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:32:21.048801 13933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:21.048847 13933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:21.048907 13933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:21.048941 13933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:21.058152 13933 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.194:41177
I20260926 16:32:21.058173 14061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.194:41177 every 8 connection(s)
I20260926 16:32:21.059209 13933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
I20260926 16:32:21.067930 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 13933
I20260926 16:32:21.068053 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance
I20260926 16:32:21.068353 11299 external_mini_cluster.cc:1428] Setting key ada9e677dbedc9be59e51f10abdf1950
Sep 26 16:32:21 dist-test-slave-2zvq krb5kdc[13692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440341, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:32:21.074187 13776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:21.061230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:41891 (local address 127.11.8.254:39733)
0926 16:32:21.061360 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:21.061364 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:21.062525 (+ 1161us) server_negotiation.cc:407] Connection header received
0926 16:32:21.063529 (+ 1004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:21.063533 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:21.063590 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:21.063698 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:21.064848 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:21.065378 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:21.066625 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:21.066856 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:21.066971 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:21.069971 (+ 3000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:21.069975 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:21.069977 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:21.070002 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:21.071912 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:21.072561 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:21.072564 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:21.072565 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:21.072639 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:21.073020 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:21.073022 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:21.073023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:21.073231 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:21.073311 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:21.073940 (+ 629us) server_negotiation.cc:299] Negotiation successful
0926 16:32:21.074043 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 16:32:21.075374 14064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:21.061630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:39733 (local address 127.11.8.194:41891)
0926 16:32:21.062326 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:21.062415 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:21.063321 (+ 906us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:21.063871 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:21.063879 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:21.064237 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:21.064702 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:21.064715 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:21.065522 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:21.065526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:21.066502 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:21.066512 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:21.067014 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:21.067018 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:21.067181 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:21.067790 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:21.067813 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:21.069791 (+ 1978us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:21.072058 (+ 2267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:21.072065 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:21.072080 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:21.072459 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:21.072730 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:21.072733 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:21.072736 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:21.072925 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:21.073343 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:21.073352 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:21.073688 (+ 336us) client_negotiation.cc:769] Sending connection context
0926 16:32:21.073898 (+ 210us) client_negotiation.cc:240] Negotiation successful
0926 16:32:21.074250 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":119,"threads_started":1}
I20260926 16:32:21.076493 14062 heartbeater.cc:347] Connected to a master server at 127.11.8.254:39733
I20260926 16:32:21.076730 14062 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:21.077220 14062 heartbeater.cc:511] Master 127.11.8.254:39733 requested a full tablet report, sending...
I20260926 16:32:21.078363 13737 ts_manager.cc:194] Registered new tserver with Master: 50c1ffe9e8184a70b25e3657c3438146 (127.11.8.194:41177)
WARNING: no policy specified for kudu/127.11.8.195@KRBTEST.COM; defaulting to no policy
I20260926 16:32:21.078989 13737 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.194@KRBTEST.COM'} at 127.11.8.194:41891
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab.
Entry for principal kudu/127.11.8.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab.
I20260926 16:32:21.095620 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.195:0
--local_ip_for_outbound_sockets=127.11.8.195
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:39733
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:43891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:21.209079 14067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:21.209403 14067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:21.209512 14067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:21.213748 14067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:21.213832 14067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:21.213917 14067 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.195
I20260926 16:32:21.219439 14067 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:43891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:39733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14067
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.195
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:21.220592 14067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:21.221652 14067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:21.232369 14075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:21.232369 14072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:21.232389 14073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:21.232589 14067 server_base.cc:1034] running on GCE node
I20260926 16:32:21.232988 14067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:21.233685 14067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:21.234849 14067 hybrid_clock.cc:648] HybridClock initialized: now 1790440341234828 us; error 33 us; skew 500 ppm
Sep 26 16:32:21 dist-test-slave-2zvq krb5kdc[13692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440341, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:21.237643 14067 init.cc:377] Logged in from keytab as kudu/127.11.8.195@KRBTEST.COM (short username kudu)
I20260926 16:32:21.238656 14067 webserver.cc:460] Webserver started at http://127.11.8.195:46579/ using document root <none> and password file <none>
I20260926 16:32:21.239212 14067 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:21.239298 14067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:21.239506 14067 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:21.241582 14067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-2/data/instance:
uuid: "a20efdc6c7b741e68c44c0a5e8cc57a3"
format_stamp: "Formatted at 2026-09-26 16:32:21 on dist-test-slave-2zvq"
server_key: "33b1d939cc2d8ee129a66b914cb575f9"
server_key_iv: "cf813a1758bc277f9c5874f5326f3edf"
server_key_version: "encryptionkey@0"
I20260926 16:32:21.242136 14067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance:
uuid: "a20efdc6c7b741e68c44c0a5e8cc57a3"
format_stamp: "Formatted at 2026-09-26 16:32:21 on dist-test-slave-2zvq"
server_key: "33b1d939cc2d8ee129a66b914cb575f9"
server_key_iv: "cf813a1758bc277f9c5874f5326f3edf"
server_key_version: "encryptionkey@0"
I20260926 16:32:21.245911 14067 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260926 16:32:21.248440 14082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:21.249655 14067 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:32:21.249775 14067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-2/wal
uuid: "a20efdc6c7b741e68c44c0a5e8cc57a3"
format_stamp: "Formatted at 2026-09-26 16:32:21 on dist-test-slave-2zvq"
server_key: "33b1d939cc2d8ee129a66b914cb575f9"
server_key_iv: "cf813a1758bc277f9c5874f5326f3edf"
server_key_version: "encryptionkey@0"
I20260926 16:32:21.249887 14067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:21.276300 14067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:21.279336 14067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:21.279538 14067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:21.280125 14067 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:32:21.281271 14067 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:21.281335 14067 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:21.281399 14067 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:21.281428 14067 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:21.290418 14067 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.195:42031
I20260926 16:32:21.290446 14195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.195:42031 every 8 connection(s)
I20260926 16:32:21.291285 14067 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
I20260926 16:32:21.291791 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 14067
I20260926 16:32:21.291908 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance
I20260926 16:32:21.292191 11299 external_mini_cluster.cc:1428] Setting key 199bf313e607a4cb038c41bb669f5fd3
Sep 26 16:32:21 dist-test-slave-2zvq krb5kdc[13692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440341, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:32:21.305011 13776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:21.293046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:46659 (local address 127.11.8.254:39733)
0926 16:32:21.293182 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:21.293187 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:21.294496 (+ 1309us) server_negotiation.cc:407] Connection header received
0926 16:32:21.295428 (+ 932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:21.295430 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:21.295484 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:21.295550 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:21.296638 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:21.297213 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:21.298579 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:21.298812 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:21.298925 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:21.301540 (+ 2615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:21.301543 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:21.301545 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:21.301571 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:21.302968 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:21.303484 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:21.303487 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:21.303489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:21.303587 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:21.303915 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:21.303920 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:21.303922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:21.304105 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:21.304207 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:21.304692 (+ 485us) server_negotiation.cc:299] Negotiation successful
0926 16:32:21.304838 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 16:32:21.305999 14198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:21.293495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:39733 (local address 127.11.8.195:46659)
0926 16:32:21.294336 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:21.294393 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:21.295242 (+ 849us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:21.295679 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:21.295686 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:21.296038 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:21.296518 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:21.296531 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:21.297337 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:21.297341 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:21.298436 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:21.298447 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:21.298991 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:21.298996 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:21.299165 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:21.299845 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:21.299861 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:21.301345 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:21.303080 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:21.303086 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:21.303103 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:21.303360 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:21.303679 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:21.303682 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:21.303684 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:21.303821 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:21.304205 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:21.304212 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:21.304464 (+ 252us) client_negotiation.cc:769] Sending connection context
0926 16:32:21.304654 (+ 190us) client_negotiation.cc:240] Negotiation successful
0926 16:32:21.305001 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":139,"threads_started":1}
I20260926 16:32:21.307123 14196 heartbeater.cc:347] Connected to a master server at 127.11.8.254:39733
I20260926 16:32:21.307417 14196 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:21.307921 14196 heartbeater.cc:511] Master 127.11.8.254:39733 requested a full tablet report, sending...
I20260926 16:32:21.308880 13737 ts_manager.cc:194] Registered new tserver with Master: a20efdc6c7b741e68c44c0a5e8cc57a3 (127.11.8.195:42031)
I20260926 16:32:21.309394 13737 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.195@KRBTEST.COM'} at 127.11.8.195:46659
I20260926 16:32:21.318576 11299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 16:32:21.331651 13776 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:21.330071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52230 (local address 127.11.8.254:39733)
0926 16:32:21.330216 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:21.330221 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:21.330468 (+ 247us) server_negotiation.cc:407] Connection header received
0926 16:32:21.330581 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:21.330584 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:21.330625 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:21.330695 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:21.331508 (+ 813us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52230: BlockingRecv error: recv got EOF from 127.0.0.1:52230 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 16:32:21.332947 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 13799
I20260926 16:32:21.339068 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 13933
I20260926 16:32:21.344638 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 14067
I20260926 16:32:21.350210 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 13705
2026-09-26T16:32:21Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2097 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:32:22 dist-test-slave-2zvq krb5kdc[14208](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:32:22 dist-test-slave-2zvq krb5kdc[14208](info): set up 2 sockets
Sep 26 16:32:22 dist-test-slave-2zvq krb5kdc[14208](info): commencing operation
krb5kdc: starting...
W20260926 16:32:23.111968 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.740s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:32:23 dist-test-slave-2zvq krb5kdc[14208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:32:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:32:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
I20260926 16:32:23.234673 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:40507
--webserver_interface=127.11.8.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:46839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:40507
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:23.347754 14221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:23.348040 14221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:23.348083 14221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:23.352366 14221 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:32:23.352429 14221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:23.352448 14221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:23.352466 14221 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:32:23.352519 14221 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:32:23.357817 14221 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:46839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:40507
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:40507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:23.358965 14221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:23.360050 14221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:23.368660 14227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:23.368722 14221 server_base.cc:1034] running on GCE node
W20260926 16:32:23.368645 14229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:23.368665 14226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:23.369457 14221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:23.370654 14221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:23.371819 14221 hybrid_clock.cc:648] HybridClock initialized: now 1790440343371776 us; error 54 us; skew 500 ppm
Sep 26 16:32:23 dist-test-slave-2zvq krb5kdc[14208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440343, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:23.381886 14221 init.cc:377] Logged in from keytab as kudu/127.11.8.254@KRBTEST.COM (short username kudu)
I20260926 16:32:23.383054 14221 webserver.cc:460] Webserver started at http://127.11.8.254:34983/ using document root <none> and password file <none>
I20260926 16:32:23.383654 14221 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:23.383720 14221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:23.383896 14221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:23.386205 14221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "9b67bff870aa404fbd380b81b5032219"
format_stamp: "Formatted at 2026-09-26 16:32:23 on dist-test-slave-2zvq"
server_key: "6d6331afafa051d888e82fb23384974c"
server_key_iv: "c88e286a124a22c061689c98f3215d1b"
server_key_version: "encryptionkey@0"
I20260926 16:32:23.386909 14221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "9b67bff870aa404fbd380b81b5032219"
format_stamp: "Formatted at 2026-09-26 16:32:23 on dist-test-slave-2zvq"
server_key: "6d6331afafa051d888e82fb23384974c"
server_key_iv: "c88e286a124a22c061689c98f3215d1b"
server_key_version: "encryptionkey@0"
I20260926 16:32:23.391098 14221 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 16:32:23.393669 14236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:23.395260 14221 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:32:23.395418 14221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "9b67bff870aa404fbd380b81b5032219"
format_stamp: "Formatted at 2026-09-26 16:32:23 on dist-test-slave-2zvq"
server_key: "6d6331afafa051d888e82fb23384974c"
server_key_iv: "c88e286a124a22c061689c98f3215d1b"
server_key_version: "encryptionkey@0"
I20260926 16:32:23.395548 14221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:23.411069 14221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:23.414093 14221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:23.414366 14221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:23.421861 14221 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:40507
I20260926 16:32:23.422257 14288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:40507 every 8 connection(s)
I20260926 16:32:23.423075 14221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:32:23.426079 14289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:23.431208 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 14221
I20260926 16:32:23.431344 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:32:23.431592 11299 external_mini_cluster.cc:1428] Setting key 47491b85858a7bf2a2c2059819aebd66
I20260926 16:32:23.431994 14289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b67bff870aa404fbd380b81b5032219: Bootstrap starting.
I20260926 16:32:23.434598 14289 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b67bff870aa404fbd380b81b5032219: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:23.435336 14289 log.cc:824] T 00000000000000000000000000000000 P 9b67bff870aa404fbd380b81b5032219: Log is configured to *not* fsync() on all Append() calls
I20260926 16:32:23.437217 14289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b67bff870aa404fbd380b81b5032219: No bootstrap required, opened a new log
Sep 26 16:32:23 dist-test-slave-2zvq krb5kdc[14208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:32:23.440778 14289 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b67bff870aa404fbd380b81b5032219 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b67bff870aa404fbd380b81b5032219" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 40507 } }
I20260926 16:32:23.441030 14289 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b67bff870aa404fbd380b81b5032219 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:23.441105 14289 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b67bff870aa404fbd380b81b5032219 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b67bff870aa404fbd380b81b5032219, State: Initialized, Role: FOLLOWER
I20260926 16:32:23.441535 14289 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b67bff870aa404fbd380b81b5032219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b67bff870aa404fbd380b81b5032219" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 40507 } }
I20260926 16:32:23.441685 14289 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b67bff870aa404fbd380b81b5032219 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:32:23.441764 14289 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b67bff870aa404fbd380b81b5032219 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:32:23.441888 14289 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b67bff870aa404fbd380b81b5032219 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:23.443054 14289 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b67bff870aa404fbd380b81b5032219 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b67bff870aa404fbd380b81b5032219" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 40507 } }
I20260926 16:32:23.443430 14289 leader_election.cc:302] T 00000000000000000000000000000000 P 9b67bff870aa404fbd380b81b5032219 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b67bff870aa404fbd380b81b5032219; no voters:
I20260926 16:32:23.443786 14289 leader_election.cc:288] T 00000000000000000000000000000000 P 9b67bff870aa404fbd380b81b5032219 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:32:23.443866 14294 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b67bff870aa404fbd380b81b5032219 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:32:23.444056 14294 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b67bff870aa404fbd380b81b5032219 [term 1 LEADER]: Becoming Leader. State: Replica: 9b67bff870aa404fbd380b81b5032219, State: Running, Role: LEADER
I20260926 16:32:23.444416 14294 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b67bff870aa404fbd380b81b5032219 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b67bff870aa404fbd380b81b5032219" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 40507 } }
I20260926 16:32:23.445113 14289 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b67bff870aa404fbd380b81b5032219 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:32:23.446179 14295 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b67bff870aa404fbd380b81b5032219 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b67bff870aa404fbd380b81b5032219" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b67bff870aa404fbd380b81b5032219" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 40507 } } }
I20260926 16:32:23.446178 14296 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b67bff870aa404fbd380b81b5032219 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b67bff870aa404fbd380b81b5032219. Latest consensus state: current_term: 1 leader_uuid: "9b67bff870aa404fbd380b81b5032219" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b67bff870aa404fbd380b81b5032219" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 40507 } } }
I20260926 16:32:23.446350 14295 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b67bff870aa404fbd380b81b5032219 [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:23.446357 14296 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b67bff870aa404fbd380b81b5032219 [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:23.446753 14292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:23.433025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49972 (local address 127.11.8.254:40507)
0926 16:32:23.433543 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:23.433559 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:23.433595 (+ 36us) server_negotiation.cc:407] Connection header received
0926 16:32:23.434226 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:23.434243 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:23.434707 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:23.435075 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:23.435900 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:23.436863 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:23.437827 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:23.438342 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:23.438531 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:23.440990 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:23.440995 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:23.441008 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:23.441057 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:23.443117 (+ 2060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:23.443621 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:23.443627 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:23.443635 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:23.443754 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:23.444200 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:23.444204 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:23.444205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:23.444706 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:23.444923 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:23.445233 (+ 310us) server_negotiation.cc:299] Negotiation successful
0926 16:32:23.445706 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":133,"threads_started":1}
I20260926 16:32:23.446911 14303 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:32:23.450117 14303 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:32:23.450126 11299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:32:23.456298 14303 catalog_manager.cc:1347] Generated new cluster ID: 8376dc8121674607be2575cb8f559657
I20260926 16:32:23.456368 14303 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:32:23.473795 14303 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:32:23.475080 14303 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:32:23.478336 14292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:23.472424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49986 (local address 127.11.8.254:40507)
0926 16:32:23.472543 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:23.472548 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:23.472840 (+ 292us) server_negotiation.cc:407] Connection header received
0926 16:32:23.472995 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:23.472999 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:23.473049 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:23.473103 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:23.473860 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:23.474635 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:23.475858 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:23.476185 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:23.476291 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:23.477248 (+ 957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:23.477251 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:23.477254 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:32:23.477846 (+ 592us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:23.477877 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:32:23.477896 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:23.477979 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:23.478104 (+ 125us) server_negotiation.cc:299] Negotiation successful
0926 16:32:23.478178 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 16:32:23.488544 14303 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b67bff870aa404fbd380b81b5032219: Generated new TSK 0
I20260926 16:32:23.489454 14303 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:32:23.498564 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 14221
2026-09-26T16:32:23Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2146 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:32:23 dist-test-slave-2zvq krb5kdc[14322](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:32:23 dist-test-slave-2zvq krb5kdc[14322](info): set up 2 sockets
Sep 26 16:32:23 dist-test-slave-2zvq krb5kdc[14322](info): commencing operation
krb5kdc: starting...
W20260926 16:32:24.539912 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:32:24 dist-test-slave-2zvq krb5kdc[14322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440344, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:32:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:32:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
I20260926 16:32:24.665654 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:44009
--webserver_interface=127.11.8.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:35129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:44009
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:24.773387 14335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:24.773633 14335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:24.773671 14335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:24.777537 14335 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:32:24.777592 14335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:24.777611 14335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:24.777628 14335 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:32:24.777642 14335 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:32:24.782820 14335 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:35129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:44009
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:44009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:24.783900 14335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:24.784893 14335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:24.793314 14340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:24.793393 14343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:24.793442 14335 server_base.cc:1034] running on GCE node
W20260926 16:32:24.793314 14341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:24.794204 14335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:24.795434 14335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:24.796636 14335 hybrid_clock.cc:648] HybridClock initialized: now 1790440344796602 us; error 44 us; skew 500 ppm
Sep 26 16:32:24 dist-test-slave-2zvq krb5kdc[14322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440344, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:24.799410 14335 init.cc:377] Logged in from keytab as kudu/127.11.8.254@KRBTEST.COM (short username kudu)
I20260926 16:32:24.800289 14335 webserver.cc:460] Webserver started at http://127.11.8.254:41371/ using document root <none> and password file <none>
I20260926 16:32:24.800866 14335 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:24.800968 14335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:24.801201 14335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:24.803328 14335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "3283bce0da43449c8de2922a0d80f565"
format_stamp: "Formatted at 2026-09-26 16:32:24 on dist-test-slave-2zvq"
server_key: "21302168e164511a80c424ce5950742e"
server_key_iv: "f6428033551521693d012787fe75aed9"
server_key_version: "encryptionkey@0"
I20260926 16:32:24.803911 14335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "3283bce0da43449c8de2922a0d80f565"
format_stamp: "Formatted at 2026-09-26 16:32:24 on dist-test-slave-2zvq"
server_key: "21302168e164511a80c424ce5950742e"
server_key_iv: "f6428033551521693d012787fe75aed9"
server_key_version: "encryptionkey@0"
I20260926 16:32:24.808022 14335 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 16:32:24.810576 14350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:24.811842 14335 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:32:24.811995 14335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "3283bce0da43449c8de2922a0d80f565"
format_stamp: "Formatted at 2026-09-26 16:32:24 on dist-test-slave-2zvq"
server_key: "21302168e164511a80c424ce5950742e"
server_key_iv: "f6428033551521693d012787fe75aed9"
server_key_version: "encryptionkey@0"
I20260926 16:32:24.812119 14335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:24.830932 14335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:24.834435 14335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:24.834650 14335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:24.841861 14335 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:44009
I20260926 16:32:24.842190 14402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:44009 every 8 connection(s)
I20260926 16:32:24.842947 14335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:32:24.845676 14403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:24.851120 14403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3283bce0da43449c8de2922a0d80f565: Bootstrap starting.
I20260926 16:32:24.851711 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 14335
I20260926 16:32:24.851866 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:32:24.852159 11299 external_mini_cluster.cc:1428] Setting key 0b1a0b42cb4e7b30aaee0ee4737a5e04
I20260926 16:32:24.853794 14403 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3283bce0da43449c8de2922a0d80f565: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:24.854528 14403 log.cc:824] T 00000000000000000000000000000000 P 3283bce0da43449c8de2922a0d80f565: Log is configured to *not* fsync() on all Append() calls
I20260926 16:32:24.856669 14403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3283bce0da43449c8de2922a0d80f565: No bootstrap required, opened a new log
I20260926 16:32:24.860584 14403 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3283bce0da43449c8de2922a0d80f565 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3283bce0da43449c8de2922a0d80f565" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 44009 } }
I20260926 16:32:24.860843 14403 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3283bce0da43449c8de2922a0d80f565 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:24.860926 14403 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3283bce0da43449c8de2922a0d80f565 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3283bce0da43449c8de2922a0d80f565, State: Initialized, Role: FOLLOWER
Sep 26 16:32:24 dist-test-slave-2zvq krb5kdc[14322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440344, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:32:24.861428 14403 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3283bce0da43449c8de2922a0d80f565 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3283bce0da43449c8de2922a0d80f565" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 44009 } }
I20260926 16:32:24.861557 14403 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3283bce0da43449c8de2922a0d80f565 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:32:24.861642 14403 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3283bce0da43449c8de2922a0d80f565 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:32:24.861768 14403 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3283bce0da43449c8de2922a0d80f565 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:24.863253 14403 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3283bce0da43449c8de2922a0d80f565 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3283bce0da43449c8de2922a0d80f565" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 44009 } }
I20260926 16:32:24.863662 14403 leader_election.cc:302] T 00000000000000000000000000000000 P 3283bce0da43449c8de2922a0d80f565 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3283bce0da43449c8de2922a0d80f565; no voters:
I20260926 16:32:24.864085 14403 leader_election.cc:288] T 00000000000000000000000000000000 P 3283bce0da43449c8de2922a0d80f565 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:32:24.864215 14408 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3283bce0da43449c8de2922a0d80f565 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:32:24.864454 14408 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3283bce0da43449c8de2922a0d80f565 [term 1 LEADER]: Becoming Leader. State: Replica: 3283bce0da43449c8de2922a0d80f565, State: Running, Role: LEADER
I20260926 16:32:24.864981 14408 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3283bce0da43449c8de2922a0d80f565 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3283bce0da43449c8de2922a0d80f565" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 44009 } }
I20260926 16:32:24.865528 14403 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3283bce0da43449c8de2922a0d80f565 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:32:24.866789 14409 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3283bce0da43449c8de2922a0d80f565 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3283bce0da43449c8de2922a0d80f565" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3283bce0da43449c8de2922a0d80f565" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 44009 } } }
I20260926 16:32:24.866914 14409 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3283bce0da43449c8de2922a0d80f565 [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:24.867213 14410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3283bce0da43449c8de2922a0d80f565 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3283bce0da43449c8de2922a0d80f565. Latest consensus state: current_term: 1 leader_uuid: "3283bce0da43449c8de2922a0d80f565" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3283bce0da43449c8de2922a0d80f565" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 44009 } } }
I20260926 16:32:24.867344 14410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3283bce0da43449c8de2922a0d80f565 [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:24.867717 14406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:24.853404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53502 (local address 127.11.8.254:44009)
0926 16:32:24.853835 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:24.853885 (+ 50us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:24.853932 (+ 47us) server_negotiation.cc:407] Connection header received
0926 16:32:24.854656 (+ 724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:24.854676 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:24.855006 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:24.855322 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:24.855984 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:24.857402 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:24.858337 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:24.859061 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:24.859246 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:24.861799 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:24.861805 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:24.861822 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:24.861860 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:24.864041 (+ 2181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:24.864613 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:24.864619 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:24.864626 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:24.864773 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:24.865097 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:24.865101 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:24.865103 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:24.865659 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:24.865842 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:24.866210 (+ 368us) server_negotiation.cc:299] Negotiation successful
0926 16:32:24.866580 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":150,"threads_started":1}
I20260926 16:32:24.867832 14417 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:32:24.871040 14417 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:32:24.876962 14417 catalog_manager.cc:1347] Generated new cluster ID: 4b2f610c5fb847599a1137e55728810c
I20260926 16:32:24.877048 14417 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:32:24.901607 14417 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:32:24.902830 14417 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:32:24.911192 14417 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3283bce0da43449c8de2922a0d80f565: Generated new TSK 0
I20260926 16:32:24.911892 14417 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.8.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab.
Entry for principal kudu/127.11.8.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab.
I20260926 16:32:24.945372 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.193:0
--local_ip_for_outbound_sockets=127.11.8.193
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:44009
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:35129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:25.055794 14429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:25.056064 14429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:25.056108 14429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:25.060158 14429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:25.060228 14429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:25.060287 14429 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.193
I20260926 16:32:25.065774 14429 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:35129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:44009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.193
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:25.067019 14429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:25.068065 14429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:25.077636 14437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:25.077697 14429 server_base.cc:1034] running on GCE node
W20260926 16:32:25.077750 14434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:25.077911 14435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:25.078375 14429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:25.078980 14429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:25.080143 14429 hybrid_clock.cc:648] HybridClock initialized: now 1790440345080122 us; error 36 us; skew 500 ppm
Sep 26 16:32:25 dist-test-slave-2zvq krb5kdc[14322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440345, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:25.083047 14429 init.cc:377] Logged in from keytab as kudu/127.11.8.193@KRBTEST.COM (short username kudu)
I20260926 16:32:25.083889 14429 webserver.cc:460] Webserver started at http://127.11.8.193:44373/ using document root <none> and password file <none>
I20260926 16:32:25.084427 14429 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:25.084523 14429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:25.084751 14429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:25.087091 14429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-0/data/instance:
uuid: "fddef1b7b05a413bb235819f4f59d34d"
format_stamp: "Formatted at 2026-09-26 16:32:25 on dist-test-slave-2zvq"
server_key: "dbc6de4725fef2bb3b89269092a3892f"
server_key_iv: "60665bbcb0f5d70a557af5099934dda4"
server_key_version: "encryptionkey@0"
I20260926 16:32:25.087695 14429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance:
uuid: "fddef1b7b05a413bb235819f4f59d34d"
format_stamp: "Formatted at 2026-09-26 16:32:25 on dist-test-slave-2zvq"
server_key: "dbc6de4725fef2bb3b89269092a3892f"
server_key_iv: "60665bbcb0f5d70a557af5099934dda4"
server_key_version: "encryptionkey@0"
I20260926 16:32:25.091542 14429 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 16:32:25.093930 14444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:25.095178 14429 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:32:25.095309 14429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-0/wal
uuid: "fddef1b7b05a413bb235819f4f59d34d"
format_stamp: "Formatted at 2026-09-26 16:32:25 on dist-test-slave-2zvq"
server_key: "dbc6de4725fef2bb3b89269092a3892f"
server_key_iv: "60665bbcb0f5d70a557af5099934dda4"
server_key_version: "encryptionkey@0"
I20260926 16:32:25.095415 14429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:25.112984 14429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:25.117015 14429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:25.117228 14429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:25.117902 14429 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:32:25.118907 14429 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:25.118959 14429 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:25.119000 14429 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:25.119057 14429 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:25.127624 14429 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.193:46173
I20260926 16:32:25.127696 14557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.193:46173 every 8 connection(s)
I20260926 16:32:25.128620 14429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
I20260926 16:32:25.131925 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 14429
I20260926 16:32:25.132085 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance
I20260926 16:32:25.132401 11299 external_mini_cluster.cc:1428] Setting key f1ecf46d0fd4d89111a30cbab889a305
Sep 26 16:32:25 dist-test-slave-2zvq krb5kdc[14322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440345, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.8.194@KRBTEST.COM; defaulting to no policy
I20260926 16:32:25.145306 14406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:25.130626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:47277 (local address 127.11.8.254:44009)
0926 16:32:25.130789 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:25.130793 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:25.131735 (+ 942us) server_negotiation.cc:407] Connection header received
0926 16:32:25.133570 (+ 1835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:25.133578 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:25.133622 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:25.133736 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:25.135132 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:25.135921 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:25.137667 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:25.137922 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:25.138023 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:25.141337 (+ 3314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:25.141341 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:25.141343 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:25.141367 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:25.143091 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:25.143658 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:25.143662 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:25.143664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:25.143752 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:25.144128 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:25.144133 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:25.144135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:25.144346 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:25.144455 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:25.144994 (+ 539us) server_negotiation.cc:299] Negotiation successful
0926 16:32:25.145124 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.194@KRBTEST.COM" created.
I20260926 16:32:25.146260 14560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:25.131072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:44009 (local address 127.11.8.193:47277)
0926 16:32:25.131573 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:25.131620 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:25.133340 (+ 1720us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:25.133988 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:25.133999 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:25.134457 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:25.134969 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:25.134983 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:25.136061 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:25.136065 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:25.137514 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:25.137529 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:25.138205 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:25.138213 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:25.138508 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:25.139215 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:25.139246 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:25.141138 (+ 1892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:25.143227 (+ 2089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:25.143233 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:25.143252 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:25.143540 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:25.143855 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:25.143859 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:25.143861 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:25.144029 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:25.144464 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:25.144473 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:25.144746 (+ 273us) client_negotiation.cc:769] Sending connection context
0926 16:32:25.144956 (+ 210us) client_negotiation.cc:240] Negotiation successful
0926 16:32:25.145306 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":123,"threads_started":1}
I20260926 16:32:25.147737 14558 heartbeater.cc:347] Connected to a master server at 127.11.8.254:44009
I20260926 16:32:25.147997 14558 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:25.148540 14558 heartbeater.cc:511] Master 127.11.8.254:44009 requested a full tablet report, sending...
I20260926 16:32:25.150151 14367 ts_manager.cc:194] Registered new tserver with Master: fddef1b7b05a413bb235819f4f59d34d (127.11.8.193:46173)
I20260926 16:32:25.151527 14367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.193@KRBTEST.COM'} at 127.11.8.193:47277
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab.
Entry for principal kudu/127.11.8.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab.
I20260926 16:32:25.162693 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.194:0
--local_ip_for_outbound_sockets=127.11.8.194
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:44009
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:35129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:25.279208 14563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:25.279467 14563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:25.279510 14563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:25.283411 14563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:25.283460 14563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:25.283517 14563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.194
I20260926 16:32:25.288658 14563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:35129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:44009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.194
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:25.289556 14563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:25.290495 14563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:25.299777 14568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:25.299830 14569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:25.299821 14571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:25.300343 14563 server_base.cc:1034] running on GCE node
I20260926 16:32:25.300707 14563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:25.301293 14563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:25.302485 14563 hybrid_clock.cc:648] HybridClock initialized: now 1790440345302472 us; error 36 us; skew 500 ppm
Sep 26 16:32:25 dist-test-slave-2zvq krb5kdc[14322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440345, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:25.305514 14563 init.cc:377] Logged in from keytab as kudu/127.11.8.194@KRBTEST.COM (short username kudu)
I20260926 16:32:25.306558 14563 webserver.cc:460] Webserver started at http://127.11.8.194:44549/ using document root <none> and password file <none>
I20260926 16:32:25.307142 14563 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:25.307233 14563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:25.307471 14563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:25.309796 14563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-1/data/instance:
uuid: "27ded1cd97b6498293a5d04fd3124b84"
format_stamp: "Formatted at 2026-09-26 16:32:25 on dist-test-slave-2zvq"
server_key: "474f895b9a4c00ac43a127baf3f07a0b"
server_key_iv: "cee4b462fb9000622bd776d525f18e48"
server_key_version: "encryptionkey@0"
I20260926 16:32:25.310447 14563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance:
uuid: "27ded1cd97b6498293a5d04fd3124b84"
format_stamp: "Formatted at 2026-09-26 16:32:25 on dist-test-slave-2zvq"
server_key: "474f895b9a4c00ac43a127baf3f07a0b"
server_key_iv: "cee4b462fb9000622bd776d525f18e48"
server_key_version: "encryptionkey@0"
I20260926 16:32:25.314380 14563 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 16:32:25.316939 14578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:25.318198 14563 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:32:25.318392 14563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-1/wal
uuid: "27ded1cd97b6498293a5d04fd3124b84"
format_stamp: "Formatted at 2026-09-26 16:32:25 on dist-test-slave-2zvq"
server_key: "474f895b9a4c00ac43a127baf3f07a0b"
server_key_iv: "cee4b462fb9000622bd776d525f18e48"
server_key_version: "encryptionkey@0"
I20260926 16:32:25.318543 14563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:25.345945 14563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:25.349308 14563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:25.349476 14563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:25.350050 14563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:32:25.350955 14563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:25.351006 14563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:25.351049 14563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:25.351066 14563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:25.360993 14563 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.194:33315
I20260926 16:32:25.361016 14691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.194:33315 every 8 connection(s)
I20260926 16:32:25.362012 14563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
I20260926 16:32:25.369434 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 14563
I20260926 16:32:25.369526 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance
I20260926 16:32:25.369791 11299 external_mini_cluster.cc:1428] Setting key 6d65a371b0662a86698b0d90d9da5021
Sep 26 16:32:25 dist-test-slave-2zvq krb5kdc[14322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440345, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:32:25.376670 14406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:25.364186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:50789 (local address 127.11.8.254:44009)
0926 16:32:25.364327 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:25.364331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:25.365121 (+ 790us) server_negotiation.cc:407] Connection header received
0926 16:32:25.366680 (+ 1559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:25.366684 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:25.366740 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:25.366862 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:25.368093 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:25.368622 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:25.369810 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:25.370063 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:25.370178 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:25.372840 (+ 2662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:25.372843 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:25.372845 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:25.372867 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:25.374412 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:25.375047 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:25.375051 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:25.375053 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:25.375149 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:25.375558 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:25.375561 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:25.375562 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:25.375725 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:25.375800 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:25.376451 (+ 651us) server_negotiation.cc:299] Negotiation successful
0926 16:32:25.376553 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 16:32:25.377895 14694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:25.364511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:44009 (local address 127.11.8.194:50789)
0926 16:32:25.364975 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:25.365017 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:25.366414 (+ 1397us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:25.367036 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:25.367043 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:25.367476 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:25.367948 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:25.367961 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:25.368747 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:25.368757 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:25.369689 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:25.369698 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:25.370212 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:25.370217 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:25.370445 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:25.371043 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:25.371060 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:25.372674 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:25.374532 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:25.374541 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:25.374559 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:25.374927 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:25.375245 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:25.375251 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:25.375255 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:25.375461 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:25.375833 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:25.375841 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:25.376185 (+ 344us) client_negotiation.cc:769] Sending connection context
0926 16:32:25.376432 (+ 247us) client_negotiation.cc:240] Negotiation successful
0926 16:32:25.376808 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":127,"threads_started":1}
I20260926 16:32:25.379217 14692 heartbeater.cc:347] Connected to a master server at 127.11.8.254:44009
WARNING: no policy specified for kudu/127.11.8.195@KRBTEST.COM; defaulting to no policy
I20260926 16:32:25.379539 14692 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:25.380051 14692 heartbeater.cc:511] Master 127.11.8.254:44009 requested a full tablet report, sending...
I20260926 16:32:25.381186 14367 ts_manager.cc:194] Registered new tserver with Master: 27ded1cd97b6498293a5d04fd3124b84 (127.11.8.194:33315)
I20260926 16:32:25.381817 14367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.194@KRBTEST.COM'} at 127.11.8.194:50789
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab.
Entry for principal kudu/127.11.8.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab.
I20260926 16:32:25.399758 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.195:0
--local_ip_for_outbound_sockets=127.11.8.195
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:44009
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:35129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:25.509624 14697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:25.510013 14697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:25.510105 14697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:25.514159 14697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:25.514241 14697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:25.514349 14697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.195
I20260926 16:32:25.519652 14697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:35129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:44009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.195
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:25.520673 14697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:25.521788 14697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:25.531066 14703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:25.531069 14705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:25.531066 14702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:25.531260 14697 server_base.cc:1034] running on GCE node
I20260926 16:32:25.531791 14697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:25.532429 14697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:25.533598 14697 hybrid_clock.cc:648] HybridClock initialized: now 1790440345533581 us; error 28 us; skew 500 ppm
Sep 26 16:32:25 dist-test-slave-2zvq krb5kdc[14322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440345, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:25.536391 14697 init.cc:377] Logged in from keytab as kudu/127.11.8.195@KRBTEST.COM (short username kudu)
I20260926 16:32:25.537161 14697 webserver.cc:460] Webserver started at http://127.11.8.195:36917/ using document root <none> and password file <none>
I20260926 16:32:25.537670 14697 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:25.537725 14697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:25.537882 14697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:25.539999 14697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-2/data/instance:
uuid: "cec9f819310241099482bf85c1ecbc88"
format_stamp: "Formatted at 2026-09-26 16:32:25 on dist-test-slave-2zvq"
server_key: "3f05c5509279bd6bb4c8a2313bb4be55"
server_key_iv: "cfe441a7b3b8c5806fb27d64eada45de"
server_key_version: "encryptionkey@0"
I20260926 16:32:25.540589 14697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance:
uuid: "cec9f819310241099482bf85c1ecbc88"
format_stamp: "Formatted at 2026-09-26 16:32:25 on dist-test-slave-2zvq"
server_key: "3f05c5509279bd6bb4c8a2313bb4be55"
server_key_iv: "cfe441a7b3b8c5806fb27d64eada45de"
server_key_version: "encryptionkey@0"
I20260926 16:32:25.544518 14697 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260926 16:32:25.546983 14712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:25.548221 14697 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:32:25.548358 14697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-2/wal
uuid: "cec9f819310241099482bf85c1ecbc88"
format_stamp: "Formatted at 2026-09-26 16:32:25 on dist-test-slave-2zvq"
server_key: "3f05c5509279bd6bb4c8a2313bb4be55"
server_key_iv: "cfe441a7b3b8c5806fb27d64eada45de"
server_key_version: "encryptionkey@0"
I20260926 16:32:25.548480 14697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:25.568588 14697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:25.571895 14697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:25.572090 14697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:25.572654 14697 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:32:25.573441 14697 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:25.573513 14697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:25.573585 14697 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:25.573642 14697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:25.583249 14697 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.195:34505
I20260926 16:32:25.583272 14825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.195:34505 every 8 connection(s)
I20260926 16:32:25.584285 14697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
I20260926 16:32:25.585937 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 14697
I20260926 16:32:25.586076 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance
I20260926 16:32:25.586427 11299 external_mini_cluster.cc:1428] Setting key 152fef7ab85397419ee2881b119e947f
Sep 26 16:32:25 dist-test-slave-2zvq krb5kdc[14322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440345, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:32:25.598372 14406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:25.586347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:58585 (local address 127.11.8.254:44009)
0926 16:32:25.586506 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:25.586513 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:25.587509 (+ 996us) server_negotiation.cc:407] Connection header received
0926 16:32:25.589170 (+ 1661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:25.589173 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:25.589218 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:25.589283 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:25.590577 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:25.591167 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:25.592329 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:25.592560 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:25.592671 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:25.595224 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:25.595227 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:25.595229 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:25.595252 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:25.596515 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:25.597005 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:25.597008 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:25.597009 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:25.597083 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:25.597371 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:25.597374 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:25.597375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:25.597562 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:25.597636 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:25.598083 (+ 447us) server_negotiation.cc:299] Negotiation successful
0926 16:32:25.598180 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 16:32:25.599316 14828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:25.586868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:44009 (local address 127.11.8.195:58585)
0926 16:32:25.587342 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:25.587396 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:25.588992 (+ 1596us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:25.589474 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:25.589481 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:25.589840 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:25.590421 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:25.590439 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:25.591297 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:25.591301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:25.592202 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:25.592210 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:25.592711 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:25.592715 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:25.592903 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:25.593525 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:25.593548 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:25.595046 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:25.596622 (+ 1576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:25.596627 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:25.596638 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:25.596885 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:25.597159 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:25.597161 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:25.597162 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:25.597289 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:25.597646 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:25.597650 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:25.597865 (+ 215us) client_negotiation.cc:769] Sending connection context
0926 16:32:25.598019 (+ 154us) client_negotiation.cc:240] Negotiation successful
0926 16:32:25.598324 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":142,"threads_started":1}
I20260926 16:32:25.600484 14826 heartbeater.cc:347] Connected to a master server at 127.11.8.254:44009
I20260926 16:32:25.600724 14826 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:25.601218 14826 heartbeater.cc:511] Master 127.11.8.254:44009 requested a full tablet report, sending...
I20260926 16:32:25.602314 14367 ts_manager.cc:194] Registered new tserver with Master: cec9f819310241099482bf85c1ecbc88 (127.11.8.195:34505)
I20260926 16:32:25.602891 14367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.195@KRBTEST.COM'} at 127.11.8.195:58585
I20260926 16:32:25.612660 11299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 16:32:25 dist-test-slave-2zvq krb5kdc[14322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440345, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 26 16:32:25 dist-test-slave-2zvq krb5kdc[14322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440345, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:32:25.638584 14406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:25.630026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53504 (local address 127.11.8.254:44009)
0926 16:32:25.630230 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:25.630235 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:25.630382 (+ 147us) server_negotiation.cc:407] Connection header received
0926 16:32:25.630582 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:25.630585 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:25.630639 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:25.630730 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:25.631455 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:25.631993 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:25.632846 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:25.633082 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:25.633185 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:25.635683 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:25.635686 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:25.635688 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:25.635712 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:25.637094 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:25.637472 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:25.637475 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:25.637476 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:25.637548 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:25.637774 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:25.637776 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:25.637777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:25.638005 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:25.638107 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:25.638369 (+ 262us) server_negotiation.cc:299] Negotiation successful
0926 16:32:25.638462 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
W20260926 16:32:25.639034 14367 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:53504
I20260926 16:32:25.639838 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 14429
I20260926 16:32:25.645237 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 14563
I20260926 16:32:25.650573 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 14697
I20260926 16:32:25.655805 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 14335
2026-09-26T16:32:25Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2159 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:32:25 dist-test-slave-2zvq krb5kdc[14838](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:32:25 dist-test-slave-2zvq krb5kdc[14838](info): set up 2 sockets
Sep 26 16:32:25 dist-test-slave-2zvq krb5kdc[14838](info): commencing operation
krb5kdc: starting...
W20260926 16:32:26.756031 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:32:26 dist-test-slave-2zvq krb5kdc[14838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:32:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:32:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
I20260926 16:32:26.879400 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:36559
--webserver_interface=127.11.8.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:35329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:36559
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:26.990473 14851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:26.990777 14851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:26.990875 14851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:26.994900 14851 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:32:26.994985 14851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:26.995023 14851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:26.995060 14851 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:32:26.995092 14851 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:32:27.000358 14851 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:35329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:36559
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:36559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:27.001394 14851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:27.002431 14851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:27.010692 14857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:27.010715 14859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:27.010721 14856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:27.010882 14851 server_base.cc:1034] running on GCE node
I20260926 16:32:27.011464 14851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:27.012544 14851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:27.013707 14851 hybrid_clock.cc:648] HybridClock initialized: now 1790440347013676 us; error 44 us; skew 500 ppm
Sep 26 16:32:27 dist-test-slave-2zvq krb5kdc[14838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440347, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:27.016436 14851 init.cc:377] Logged in from keytab as kudu/127.11.8.254@KRBTEST.COM (short username kudu)
I20260926 16:32:27.017261 14851 webserver.cc:460] Webserver started at http://127.11.8.254:36515/ using document root <none> and password file <none>
I20260926 16:32:27.017786 14851 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:27.017844 14851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:27.018054 14851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:27.020056 14851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "5829354a3510418eb5bbf2687e7142c1"
format_stamp: "Formatted at 2026-09-26 16:32:27 on dist-test-slave-2zvq"
server_key: "8319944ce90680c158c44de3227b5299"
server_key_iv: "e658ecf5b381756f426b7b0ff4621ddc"
server_key_version: "encryptionkey@0"
I20260926 16:32:27.020730 14851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "5829354a3510418eb5bbf2687e7142c1"
format_stamp: "Formatted at 2026-09-26 16:32:27 on dist-test-slave-2zvq"
server_key: "8319944ce90680c158c44de3227b5299"
server_key_iv: "e658ecf5b381756f426b7b0ff4621ddc"
server_key_version: "encryptionkey@0"
I20260926 16:32:27.025538 14851 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 16:32:27.028074 14866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:27.029400 14851 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:32:27.029559 14851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "5829354a3510418eb5bbf2687e7142c1"
format_stamp: "Formatted at 2026-09-26 16:32:27 on dist-test-slave-2zvq"
server_key: "8319944ce90680c158c44de3227b5299"
server_key_iv: "e658ecf5b381756f426b7b0ff4621ddc"
server_key_version: "encryptionkey@0"
I20260926 16:32:27.029685 14851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:27.047550 14851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:27.050486 14851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:27.050690 14851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:27.058084 14851 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:36559
I20260926 16:32:27.058103 14918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:36559 every 8 connection(s)
I20260926 16:32:27.059317 14851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:32:27.062175 14919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:27.065881 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 14851
I20260926 16:32:27.066035 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:32:27.066336 11299 external_mini_cluster.cc:1428] Setting key a933be66c32caaeb72ee67c9085178b3
I20260926 16:32:27.068405 14919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5829354a3510418eb5bbf2687e7142c1: Bootstrap starting.
I20260926 16:32:27.070977 14919 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5829354a3510418eb5bbf2687e7142c1: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:27.071727 14919 log.cc:824] T 00000000000000000000000000000000 P 5829354a3510418eb5bbf2687e7142c1: Log is configured to *not* fsync() on all Append() calls
I20260926 16:32:27.073938 14919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5829354a3510418eb5bbf2687e7142c1: No bootstrap required, opened a new log
Sep 26 16:32:27 dist-test-slave-2zvq krb5kdc[14838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:32:27.078107 14919 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5829354a3510418eb5bbf2687e7142c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5829354a3510418eb5bbf2687e7142c1" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 36559 } }
I20260926 16:32:27.078413 14919 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5829354a3510418eb5bbf2687e7142c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:27.078493 14919 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5829354a3510418eb5bbf2687e7142c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5829354a3510418eb5bbf2687e7142c1, State: Initialized, Role: FOLLOWER
I20260926 16:32:27.078963 14919 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5829354a3510418eb5bbf2687e7142c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5829354a3510418eb5bbf2687e7142c1" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 36559 } }
I20260926 16:32:27.079113 14919 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5829354a3510418eb5bbf2687e7142c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:32:27.079205 14919 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5829354a3510418eb5bbf2687e7142c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:32:27.079329 14919 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5829354a3510418eb5bbf2687e7142c1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:27.082229 14922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:27.067878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34012 (local address 127.11.8.254:36559)
0926 16:32:27.068302 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:27.068321 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:27.068366 (+ 45us) server_negotiation.cc:407] Connection header received
0926 16:32:27.069083 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:27.069102 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:27.069445 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:27.069735 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:27.070737 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:27.071853 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:27.073000 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:27.073503 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:27.073624 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:27.076534 (+ 2910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:27.076539 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:27.076553 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:27.076580 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:27.079320 (+ 2740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:27.079828 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:27.079834 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:27.079840 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:27.079965 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:27.080264 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:27.080268 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:27.080270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:27.080707 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:27.080946 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:27.081186 (+ 240us) server_negotiation.cc:299] Negotiation successful
0926 16:32:27.081474 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":152,"threads_started":1}
I20260926 16:32:27.097689 14919 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5829354a3510418eb5bbf2687e7142c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5829354a3510418eb5bbf2687e7142c1" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 36559 } }
I20260926 16:32:27.098198 14919 leader_election.cc:302] T 00000000000000000000000000000000 P 5829354a3510418eb5bbf2687e7142c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5829354a3510418eb5bbf2687e7142c1; no voters:
I20260926 16:32:27.098676 14919 leader_election.cc:288] T 00000000000000000000000000000000 P 5829354a3510418eb5bbf2687e7142c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:32:27.098860 14924 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5829354a3510418eb5bbf2687e7142c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:32:27.099102 14924 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5829354a3510418eb5bbf2687e7142c1 [term 1 LEADER]: Becoming Leader. State: Replica: 5829354a3510418eb5bbf2687e7142c1, State: Running, Role: LEADER
I20260926 16:32:27.099491 14924 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5829354a3510418eb5bbf2687e7142c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5829354a3510418eb5bbf2687e7142c1" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 36559 } }
I20260926 16:32:27.100173 14919 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5829354a3510418eb5bbf2687e7142c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:32:27.101132 14925 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5829354a3510418eb5bbf2687e7142c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5829354a3510418eb5bbf2687e7142c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5829354a3510418eb5bbf2687e7142c1" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 36559 } } }
I20260926 16:32:27.101254 14925 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5829354a3510418eb5bbf2687e7142c1 [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:27.101728 14926 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5829354a3510418eb5bbf2687e7142c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5829354a3510418eb5bbf2687e7142c1. Latest consensus state: current_term: 1 leader_uuid: "5829354a3510418eb5bbf2687e7142c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5829354a3510418eb5bbf2687e7142c1" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 36559 } } }
I20260926 16:32:27.101913 14926 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5829354a3510418eb5bbf2687e7142c1 [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:27.101945 14933 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:32:27.105720 14933 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:32:27.111856 14933 catalog_manager.cc:1347] Generated new cluster ID: 3348e53957a548d1b77c1c73ec962076
I20260926 16:32:27.111931 14933 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:32:27.126384 14933 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:32:27.127343 14933 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:32:27.133579 14933 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5829354a3510418eb5bbf2687e7142c1: Generated new TSK 0
I20260926 16:32:27.134490 14933 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.8.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab.
Entry for principal kudu/127.11.8.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab.
W20260926 16:32:28.303786 11299 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.11.8.193: real 1.166s user 0.000s sys 0.003s
I20260926 16:32:28.304325 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.193:0
--local_ip_for_outbound_sockets=127.11.8.193
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:36559
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:35329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:28.418530 14945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:28.418768 14945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:28.418829 14945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:28.422961 14945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:28.423025 14945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:28.423110 14945 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.193
I20260926 16:32:28.428689 14945 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:35329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:36559
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14945
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.193
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:28.429857 14945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:28.431010 14945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:28.441035 14953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:28.441039 14950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:28.441035 14951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:28.441344 14945 server_base.cc:1034] running on GCE node
I20260926 16:32:28.441820 14945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:28.442598 14945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:28.443779 14945 hybrid_clock.cc:648] HybridClock initialized: now 1790440348443740 us; error 51 us; skew 500 ppm
Sep 26 16:32:28 dist-test-slave-2zvq krb5kdc[14838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440348, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:28.454928 14945 init.cc:377] Logged in from keytab as kudu/127.11.8.193@KRBTEST.COM (short username kudu)
I20260926 16:32:28.455909 14945 webserver.cc:460] Webserver started at http://127.11.8.193:39289/ using document root <none> and password file <none>
I20260926 16:32:28.456503 14945 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:28.456566 14945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:28.456777 14945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:28.458940 14945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-0/data/instance:
uuid: "b0c48a9a8b8e4e138f4b4adf662033b0"
format_stamp: "Formatted at 2026-09-26 16:32:28 on dist-test-slave-2zvq"
server_key: "d248666d4adb3b09e3428b0d09928a00"
server_key_iv: "6fca08ac21206051231d51ccc23437f9"
server_key_version: "encryptionkey@0"
I20260926 16:32:28.459517 14945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance:
uuid: "b0c48a9a8b8e4e138f4b4adf662033b0"
format_stamp: "Formatted at 2026-09-26 16:32:28 on dist-test-slave-2zvq"
server_key: "d248666d4adb3b09e3428b0d09928a00"
server_key_iv: "6fca08ac21206051231d51ccc23437f9"
server_key_version: "encryptionkey@0"
I20260926 16:32:28.463589 14945 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 16:32:28.466380 14960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:28.467717 14945 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:32:28.467859 14945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-0/wal
uuid: "b0c48a9a8b8e4e138f4b4adf662033b0"
format_stamp: "Formatted at 2026-09-26 16:32:28 on dist-test-slave-2zvq"
server_key: "d248666d4adb3b09e3428b0d09928a00"
server_key_iv: "6fca08ac21206051231d51ccc23437f9"
server_key_version: "encryptionkey@0"
I20260926 16:32:28.467981 14945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:28.479306 14945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:28.482436 14945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:28.482641 14945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:28.483223 14945 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:32:28.484099 14945 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:28.484145 14945 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:28.484212 14945 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:28.484243 14945 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:28.493517 14945 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.193:39605
I20260926 16:32:28.493549 15073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.193:39605 every 8 connection(s)
I20260926 16:32:28.494522 14945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
I20260926 16:32:28.501118 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 14945
I20260926 16:32:28.501266 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance
I20260926 16:32:28.501631 11299 external_mini_cluster.cc:1428] Setting key f8624c4760f11123c968a12723b8a02a
Sep 26 16:32:28 dist-test-slave-2zvq krb5kdc[14838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440348, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.8.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.194@KRBTEST.COM" created.
I20260926 16:32:28.517230 15076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:28.496511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:50829 (local address 127.11.8.254:36559)
0926 16:32:28.496821 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:28.496859 (+ 38us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:28.497615 (+ 756us) server_negotiation.cc:407] Connection header received
0926 16:32:28.499165 (+ 1550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:28.499168 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:28.499217 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:28.499308 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:28.500606 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:28.501424 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:28.503013 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:28.503318 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:28.503439 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:28.507792 (+ 4353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:28.507800 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:28.507806 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:28.507844 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:28.512912 (+ 5068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:28.513545 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:28.513549 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:28.513551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:28.513769 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:28.514511 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:28.514515 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:28.514517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:28.514814 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:28.514918 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:28.516755 (+ 1837us) server_negotiation.cc:299] Negotiation successful
0926 16:32:28.517049 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":108,"threads_started":1}
I20260926 16:32:28.518169 15077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:28.496970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36559 (local address 127.11.8.193:50829)
0926 16:32:28.497453 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:28.497497 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:28.498940 (+ 1443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:28.499473 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:28.499480 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:28.499934 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:28.500432 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:28.500446 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:28.501610 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:28.501614 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:28.502874 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:28.502885 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:28.503511 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:28.503516 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:28.503697 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:28.504383 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:28.504402 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:28.507505 (+ 3103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:28.513079 (+ 5574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:28.513087 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:28.513106 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:28.513404 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:28.513871 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:28.513874 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:28.513876 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:28.514331 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:28.516210 (+ 1879us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:28.516217 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:28.516493 (+ 276us) client_negotiation.cc:769] Sending connection context
0926 16:32:28.516699 (+ 206us) client_negotiation.cc:240] Negotiation successful
0926 16:32:28.517297 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":109,"threads_started":1}
I20260926 16:32:28.520290 15074 heartbeater.cc:347] Connected to a master server at 127.11.8.254:36559
I20260926 16:32:28.520556 15074 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:28.521265 15074 heartbeater.cc:511] Master 127.11.8.254:36559 requested a full tablet report, sending...
I20260926 16:32:28.523272 14883 ts_manager.cc:194] Registered new tserver with Master: b0c48a9a8b8e4e138f4b4adf662033b0 (127.11.8.193:39605)
I20260926 16:32:28.525012 14883 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.193@KRBTEST.COM'} at 127.11.8.193:50829
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab.
Entry for principal kudu/127.11.8.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab.
I20260926 16:32:28.534855 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.194:0
--local_ip_for_outbound_sockets=127.11.8.194
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:36559
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:35329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:28.654052 15080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:28.654327 15080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:28.654369 15080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:28.658398 15080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:28.658459 15080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:28.658519 15080 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.194
I20260926 16:32:28.663909 15080 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:35329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:36559
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.194
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:28.665066 15080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:28.666165 15080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:28.675617 15085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:28.675725 15088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:28.675617 15086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:28.675902 15080 server_base.cc:1034] running on GCE node
I20260926 16:32:28.676262 15080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:28.676898 15080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:28.678095 15080 hybrid_clock.cc:648] HybridClock initialized: now 1790440348678061 us; error 45 us; skew 500 ppm
Sep 26 16:32:28 dist-test-slave-2zvq krb5kdc[14838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440348, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:28.680817 15080 init.cc:377] Logged in from keytab as kudu/127.11.8.194@KRBTEST.COM (short username kudu)
I20260926 16:32:28.681638 15080 webserver.cc:460] Webserver started at http://127.11.8.194:41807/ using document root <none> and password file <none>
I20260926 16:32:28.682132 15080 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:28.682186 15080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:28.682417 15080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:28.684321 15080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-1/data/instance:
uuid: "3c3af3c96a7b4b2aa93447a0aeba3b5c"
format_stamp: "Formatted at 2026-09-26 16:32:28 on dist-test-slave-2zvq"
server_key: "c7551c4986c2a523fc3d35ba93365632"
server_key_iv: "e42f7f98a6aff69505bc35c1307db007"
server_key_version: "encryptionkey@0"
I20260926 16:32:28.684823 15080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance:
uuid: "3c3af3c96a7b4b2aa93447a0aeba3b5c"
format_stamp: "Formatted at 2026-09-26 16:32:28 on dist-test-slave-2zvq"
server_key: "c7551c4986c2a523fc3d35ba93365632"
server_key_iv: "e42f7f98a6aff69505bc35c1307db007"
server_key_version: "encryptionkey@0"
I20260926 16:32:28.688370 15080 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260926 16:32:28.690645 15095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:28.691892 15080 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:32:28.692044 15080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-1/wal
uuid: "3c3af3c96a7b4b2aa93447a0aeba3b5c"
format_stamp: "Formatted at 2026-09-26 16:32:28 on dist-test-slave-2zvq"
server_key: "c7551c4986c2a523fc3d35ba93365632"
server_key_iv: "e42f7f98a6aff69505bc35c1307db007"
server_key_version: "encryptionkey@0"
I20260926 16:32:28.692147 15080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:28.706030 15080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:28.709236 15080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:28.709437 15080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:28.710029 15080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:32:28.710893 15080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:28.710947 15080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:28.711012 15080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:28.711051 15080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:28.720028 15080 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.194:44519
I20260926 16:32:28.720050 15208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.194:44519 every 8 connection(s)
I20260926 16:32:28.720858 15080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
I20260926 16:32:28.721836 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 15080
I20260926 16:32:28.721956 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance
I20260926 16:32:28.722290 11299 external_mini_cluster.cc:1428] Setting key ed7f3663ace88f09d6171f90b91c7c18
Sep 26 16:32:28 dist-test-slave-2zvq krb5kdc[14838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440348, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.8.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.195@KRBTEST.COM" created.
I20260926 16:32:28.735616 15076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:28.722662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:58329 (local address 127.11.8.254:36559)
0926 16:32:28.722807 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:28.722812 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:28.723737 (+ 925us) server_negotiation.cc:407] Connection header received
0926 16:32:28.725211 (+ 1474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:28.725214 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:28.725269 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:28.725338 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:28.726853 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:28.727769 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:28.728875 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:28.729322 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:28.729438 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:28.731997 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:28.732000 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:28.732002 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:28.732024 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:28.733432 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:28.734054 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:28.734056 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:28.734057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:28.734133 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:28.734510 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:28.734514 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:28.734516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:28.734704 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:28.734815 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:28.735382 (+ 567us) server_negotiation.cc:299] Negotiation successful
0926 16:32:28.735496 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 16:32:28.736688 15211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:28.723091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36559 (local address 127.11.8.194:58329)
0926 16:32:28.723589 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:28.723646 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:28.724932 (+ 1286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:28.725552 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:28.725563 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:28.726089 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:28.726707 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:28.726721 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:28.727893 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:28.727895 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:28.728749 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:28.728757 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:28.729462 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:28.729466 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:28.729600 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:28.730204 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:28.730220 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:28.731842 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:28.733535 (+ 1693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:28.733540 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:28.733551 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:28.733905 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:28.734228 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:28.734232 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:28.734234 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:28.734401 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:28.734811 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:28.734817 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:28.735059 (+ 242us) client_negotiation.cc:769] Sending connection context
0926 16:32:28.735299 (+ 240us) client_negotiation.cc:240] Negotiation successful
0926 16:32:28.735661 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":118,"threads_started":1}
I20260926 16:32:28.737826 15209 heartbeater.cc:347] Connected to a master server at 127.11.8.254:36559
I20260926 16:32:28.738089 15209 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:28.738626 15209 heartbeater.cc:511] Master 127.11.8.254:36559 requested a full tablet report, sending...
I20260926 16:32:28.739719 14883 ts_manager.cc:194] Registered new tserver with Master: 3c3af3c96a7b4b2aa93447a0aeba3b5c (127.11.8.194:44519)
I20260926 16:32:28.740561 14883 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.194@KRBTEST.COM'} at 127.11.8.194:58329
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab.
Entry for principal kudu/127.11.8.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab.
I20260926 16:32:28.749432 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.195:0
--local_ip_for_outbound_sockets=127.11.8.195
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:36559
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:35329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:28.857535 15214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:28.857762 15214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:28.857801 15214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:28.861758 15214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:28.861824 15214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:28.861919 15214 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.195
I20260926 16:32:28.867137 15214 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:35329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:36559
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15214
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.195
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:28.868242 15214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:28.869283 15214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:28.878881 15222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:28.878840 15219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:28.878840 15220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:28.879046 15214 server_base.cc:1034] running on GCE node
I20260926 16:32:28.879469 15214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:28.880065 15214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:28.881276 15214 hybrid_clock.cc:648] HybridClock initialized: now 1790440348881247 us; error 46 us; skew 500 ppm
Sep 26 16:32:28 dist-test-slave-2zvq krb5kdc[14838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440348, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:28.884182 15214 init.cc:377] Logged in from keytab as kudu/127.11.8.195@KRBTEST.COM (short username kudu)
I20260926 16:32:28.885006 15214 webserver.cc:460] Webserver started at http://127.11.8.195:39027/ using document root <none> and password file <none>
I20260926 16:32:28.885504 15214 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:28.885561 15214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:28.885722 15214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:28.887703 15214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-2/data/instance:
uuid: "3a395f5d41f04910b44e753f3693ccd9"
format_stamp: "Formatted at 2026-09-26 16:32:28 on dist-test-slave-2zvq"
server_key: "b39c331748d6c9a96a73d70057b0f6f8"
server_key_iv: "d53643ef5db78a8bfb9c16c54ffcfd5e"
server_key_version: "encryptionkey@0"
I20260926 16:32:28.888206 15214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance:
uuid: "3a395f5d41f04910b44e753f3693ccd9"
format_stamp: "Formatted at 2026-09-26 16:32:28 on dist-test-slave-2zvq"
server_key: "b39c331748d6c9a96a73d70057b0f6f8"
server_key_iv: "d53643ef5db78a8bfb9c16c54ffcfd5e"
server_key_version: "encryptionkey@0"
I20260926 16:32:28.891706 15214 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 16:32:28.893919 15229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:28.895148 15214 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:32:28.895239 15214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-2/wal
uuid: "3a395f5d41f04910b44e753f3693ccd9"
format_stamp: "Formatted at 2026-09-26 16:32:28 on dist-test-slave-2zvq"
server_key: "b39c331748d6c9a96a73d70057b0f6f8"
server_key_iv: "d53643ef5db78a8bfb9c16c54ffcfd5e"
server_key_version: "encryptionkey@0"
I20260926 16:32:28.895319 15214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:28.909690 15214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:28.912838 15214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:28.912998 15214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:28.913523 15214 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:32:28.914309 15214 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:28.914353 15214 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:28.914391 15214 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:28.914414 15214 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:28.923215 15214 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.195:33203
I20260926 16:32:28.923251 15342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.195:33203 every 8 connection(s)
I20260926 16:32:28.924068 15214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
I20260926 16:32:28.925370 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 15214
I20260926 16:32:28.925496 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance
I20260926 16:32:28.925781 11299 external_mini_cluster.cc:1428] Setting key 99b6193d62fce3834059fd2a7d9adcd2
Sep 26 16:32:28 dist-test-slave-2zvq krb5kdc[14838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440348, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:32:28.937650 15076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:28.926079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:57979 (local address 127.11.8.254:36559)
0926 16:32:28.926223 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:28.926228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:28.927185 (+ 957us) server_negotiation.cc:407] Connection header received
0926 16:32:28.928663 (+ 1478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:28.928666 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:28.928712 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:28.928791 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:28.929900 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:28.930543 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:28.931724 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:28.931974 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:28.932062 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:28.934642 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:28.934646 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:28.934648 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:28.934672 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:28.935951 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:28.936382 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:28.936385 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:28.936386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:28.936461 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:28.936735 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:28.936737 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:28.936738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:28.936909 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:28.937018 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:28.937425 (+ 407us) server_negotiation.cc:299] Negotiation successful
0926 16:32:28.937522 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 16:32:28.938464 15345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:28.926541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36559 (local address 127.11.8.195:57979)
0926 16:32:28.926999 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:28.927073 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:28.928450 (+ 1377us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:28.928917 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:28.928923 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:28.929270 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:28.929746 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:28.929759 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:28.930675 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:28.930678 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:28.931605 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:28.931614 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:28.932101 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:28.932104 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:28.932242 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:28.932845 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:28.932861 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:28.934450 (+ 1589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:28.936050 (+ 1600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:28.936054 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:28.936065 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:28.936296 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:28.936538 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:28.936540 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:28.936541 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:28.936657 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:28.937021 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:28.937026 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:28.937228 (+ 202us) client_negotiation.cc:769] Sending connection context
0926 16:32:28.937372 (+ 144us) client_negotiation.cc:240] Negotiation successful
0926 16:32:28.937610 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":116,"threads_started":1}
I20260926 16:32:28.939556 15343 heartbeater.cc:347] Connected to a master server at 127.11.8.254:36559
I20260926 16:32:28.939803 15343 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:28.940279 15343 heartbeater.cc:511] Master 127.11.8.254:36559 requested a full tablet report, sending...
I20260926 16:32:28.941268 14883 ts_manager.cc:194] Registered new tserver with Master: 3a395f5d41f04910b44e753f3693ccd9 (127.11.8.195:33203)
I20260926 16:32:28.941834 14883 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.195@KRBTEST.COM'} at 127.11.8.195:57979
I20260926 16:32:28.952183 11299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 16:32:28 dist-test-slave-2zvq krb5kdc[14838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 16:32:28 dist-test-slave-2zvq krb5kdc[14838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.193@KRBTEST.COM
I20260926 16:32:28.978654 15350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:28.968848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52680 (local address 127.11.8.193:39605)
0926 16:32:28.969200 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:28.969214 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:28.969262 (+ 48us) server_negotiation.cc:407] Connection header received
0926 16:32:28.969314 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:28.969320 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:28.969525 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:28.969646 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:28.970412 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:28.971297 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:28.972216 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:28.972665 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:28.972756 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:28.975243 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:28.975251 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:28.975264 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:28.975286 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:28.976946 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:28.977416 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:28.977421 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:28.977422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:28.977498 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:28.977743 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:28.977746 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:28.977747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:28.977999 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:28.978146 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:28.978372 (+ 226us) server_negotiation.cc:299] Negotiation successful
0926 16:32:28.978494 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":126,"threads_started":1}
Sep 26 16:32:28 dist-test-slave-2zvq krb5kdc[14838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:32:28.989709 15076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:28.981413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45118 (local address 127.11.8.254:36559)
0926 16:32:28.981563 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:28.981570 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:28.981586 (+ 16us) server_negotiation.cc:407] Connection header received
0926 16:32:28.981734 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:28.981737 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:28.981786 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:28.981911 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:28.982515 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:28.983029 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:28.983838 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:28.984064 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:28.984174 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:28.986707 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:28.986711 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:28.986712 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:28.986738 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:28.987979 (+ 1241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:28.988391 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:28.988394 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:28.988396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:28.988480 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:28.988757 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:28.988760 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:28.988762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:28.989042 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:28.989184 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:28.989423 (+ 239us) server_negotiation.cc:299] Negotiation successful
0926 16:32:28.989542 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260926 16:32:28.990314 14883 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45118
I20260926 16:32:28.991067 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 14945
I20260926 16:32:28.997068 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 15080
I20260926 16:32:29.002722 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 15214
I20260926 16:32:29.008162 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 14851
2026-09-26T16:32:29Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3352 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:32:29 dist-test-slave-2zvq krb5kdc[15355](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:32:29 dist-test-slave-2zvq krb5kdc[15355](info): set up 2 sockets
Sep 26 16:32:29 dist-test-slave-2zvq krb5kdc[15355](info): commencing operation
krb5kdc: starting...
W20260926 16:32:30.040163 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.009s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:32:30 dist-test-slave-2zvq krb5kdc[15355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440350, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:32:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:32:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
I20260926 16:32:30.161048 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:46615
--webserver_interface=127.11.8.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:46813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:46615
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:30.268783 15368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:30.269055 15368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:30.269098 15368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:30.273046 15368 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:32:30.273092 15368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:30.273110 15368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:30.273126 15368 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:32:30.273141 15368 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:32:30.278200 15368 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:46813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:46615
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:46615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:30.279310 15368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:30.280325 15368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:30.288479 15373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:30.288637 15368 server_base.cc:1034] running on GCE node
W20260926 16:32:30.288509 15376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:30.288486 15374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:30.289247 15368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:30.290416 15368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:30.291611 15368 hybrid_clock.cc:648] HybridClock initialized: now 1790440350291595 us; error 34 us; skew 500 ppm
Sep 26 16:32:30 dist-test-slave-2zvq krb5kdc[15355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440350, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:30.294239 15368 init.cc:377] Logged in from keytab as kudu/127.11.8.254@KRBTEST.COM (short username kudu)
I20260926 16:32:30.294997 15368 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:30.295092 15368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:30.295323 15368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:30.297353 15368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "fc3725660bc94165966a5e5a51d5a59f"
format_stamp: "Formatted at 2026-09-26 16:32:30 on dist-test-slave-2zvq"
server_key: "a038480cc79bc0dcfbc8dc0d301443d5"
server_key_iv: "966ecdaa120359541de1d2e56b8a83c2"
server_key_version: "encryptionkey@0"
I20260926 16:32:30.297935 15368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "fc3725660bc94165966a5e5a51d5a59f"
format_stamp: "Formatted at 2026-09-26 16:32:30 on dist-test-slave-2zvq"
server_key: "a038480cc79bc0dcfbc8dc0d301443d5"
server_key_iv: "966ecdaa120359541de1d2e56b8a83c2"
server_key_version: "encryptionkey@0"
I20260926 16:32:30.301649 15368 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260926 16:32:30.304390 15382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:30.305853 15368 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260926 16:32:30.306025 15368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "fc3725660bc94165966a5e5a51d5a59f"
format_stamp: "Formatted at 2026-09-26 16:32:30 on dist-test-slave-2zvq"
server_key: "a038480cc79bc0dcfbc8dc0d301443d5"
server_key_iv: "966ecdaa120359541de1d2e56b8a83c2"
server_key_version: "encryptionkey@0"
I20260926 16:32:30.306191 15368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:30.330106 15368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:30.333319 15368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:30.333550 15368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:30.340298 15368 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:46615
I20260926 16:32:30.340559 15434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:46615 every 8 connection(s)
I20260926 16:32:30.340962 15368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:32:30.343668 15435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:30.346903 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 15368
I20260926 16:32:30.347079 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:32:30.347340 11299 external_mini_cluster.cc:1428] Setting key 8a126226edb1eaf6d1e2f6271a3e69ff
I20260926 16:32:30.349824 15435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc3725660bc94165966a5e5a51d5a59f: Bootstrap starting.
I20260926 16:32:30.352514 15435 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc3725660bc94165966a5e5a51d5a59f: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:30.353355 15435 log.cc:824] T 00000000000000000000000000000000 P fc3725660bc94165966a5e5a51d5a59f: Log is configured to *not* fsync() on all Append() calls
I20260926 16:32:30.355504 15435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc3725660bc94165966a5e5a51d5a59f: No bootstrap required, opened a new log
Sep 26 16:32:30 dist-test-slave-2zvq krb5kdc[15355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440350, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:32:30.359371 15435 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc3725660bc94165966a5e5a51d5a59f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc3725660bc94165966a5e5a51d5a59f" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 46615 } }
I20260926 16:32:30.359579 15435 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc3725660bc94165966a5e5a51d5a59f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:30.359613 15435 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc3725660bc94165966a5e5a51d5a59f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc3725660bc94165966a5e5a51d5a59f, State: Initialized, Role: FOLLOWER
I20260926 16:32:30.360090 15435 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc3725660bc94165966a5e5a51d5a59f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc3725660bc94165966a5e5a51d5a59f" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 46615 } }
I20260926 16:32:30.360209 15435 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc3725660bc94165966a5e5a51d5a59f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:32:30.360289 15435 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc3725660bc94165966a5e5a51d5a59f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:32:30.360410 15435 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc3725660bc94165966a5e5a51d5a59f [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:30.361255 15435 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc3725660bc94165966a5e5a51d5a59f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc3725660bc94165966a5e5a51d5a59f" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 46615 } }
I20260926 16:32:30.361595 15435 leader_election.cc:302] T 00000000000000000000000000000000 P fc3725660bc94165966a5e5a51d5a59f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc3725660bc94165966a5e5a51d5a59f; no voters:
I20260926 16:32:30.361956 15435 leader_election.cc:288] T 00000000000000000000000000000000 P fc3725660bc94165966a5e5a51d5a59f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:32:30.362022 15440 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc3725660bc94165966a5e5a51d5a59f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:32:30.362215 15440 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc3725660bc94165966a5e5a51d5a59f [term 1 LEADER]: Becoming Leader. State: Replica: fc3725660bc94165966a5e5a51d5a59f, State: Running, Role: LEADER
I20260926 16:32:30.362534 15440 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc3725660bc94165966a5e5a51d5a59f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc3725660bc94165966a5e5a51d5a59f" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 46615 } }
I20260926 16:32:30.362898 15438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:30.348783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59354 (local address 127.11.8.254:46615)
0926 16:32:30.349219 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:30.349238 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:30.349279 (+ 41us) server_negotiation.cc:407] Connection header received
0926 16:32:30.350108 (+ 829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:30.350132 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:30.350574 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:30.350937 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:30.351579 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:30.353085 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:30.354173 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:30.354869 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:30.355023 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:30.357733 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:30.357738 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:30.357750 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:30.357776 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:30.359677 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:30.360141 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:30.360147 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:30.360153 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:30.360319 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:30.360633 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:30.360637 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:30.360639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:30.361032 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:30.361239 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:30.361534 (+ 295us) server_negotiation.cc:299] Negotiation successful
0926 16:32:30.361845 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":120,"threads_started":1}
I20260926 16:32:30.363209 15435 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc3725660bc94165966a5e5a51d5a59f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:32:30.363695 15440 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc3725660bc94165966a5e5a51d5a59f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc3725660bc94165966a5e5a51d5a59f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc3725660bc94165966a5e5a51d5a59f" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 46615 } } }
I20260926 16:32:30.363799 15440 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc3725660bc94165966a5e5a51d5a59f [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:30.364120 15441 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc3725660bc94165966a5e5a51d5a59f [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc3725660bc94165966a5e5a51d5a59f. Latest consensus state: current_term: 1 leader_uuid: "fc3725660bc94165966a5e5a51d5a59f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc3725660bc94165966a5e5a51d5a59f" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 46615 } } }
I20260926 16:32:30.364209 15445 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:32:30.364210 15441 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc3725660bc94165966a5e5a51d5a59f [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:30.367365 15445 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:32:30.372493 15445 catalog_manager.cc:1347] Generated new cluster ID: 58bd928a44704bab970e0c8a3fc65c23
I20260926 16:32:30.372550 15445 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:32:30.386684 15445 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:32:30.387761 15445 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:32:30.392400 15445 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc3725660bc94165966a5e5a51d5a59f: Generated new TSK 0
I20260926 16:32:30.393131 15445 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.8.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab.
Entry for principal kudu/127.11.8.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab.
I20260926 16:32:30.431375 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.193:0
--local_ip_for_outbound_sockets=127.11.8.193
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:46615
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:46813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:30.548969 15461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:30.549234 15461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:30.549279 15461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:30.553316 15461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:30.553380 15461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:30.553439 15461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.193
I20260926 16:32:30.558633 15461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:46813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:46615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.193
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:30.559599 15461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:30.560544 15461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 16:32:30.569774 15461 server_base.cc:1034] running on GCE node
W20260926 16:32:30.569726 15466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:30.569742 15467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:30.569726 15469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:30.570396 15461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:30.570973 15461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:30.572135 15461 hybrid_clock.cc:648] HybridClock initialized: now 1790440350572123 us; error 21 us; skew 500 ppm
Sep 26 16:32:30 dist-test-slave-2zvq krb5kdc[15355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440350, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:30.574934 15461 init.cc:377] Logged in from keytab as kudu/127.11.8.193@KRBTEST.COM (short username kudu)
I20260926 16:32:30.575644 15461 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:30.575704 15461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:30.575915 15461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:30.577968 15461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-0/data/instance:
uuid: "fba47c89bf8d4b6599b46a5f22ed7683"
format_stamp: "Formatted at 2026-09-26 16:32:30 on dist-test-slave-2zvq"
server_key: "9f5bd39a10c62a78408f9d74122e0767"
server_key_iv: "b72aa6fe120f12ca6c276c518e020ef0"
server_key_version: "encryptionkey@0"
I20260926 16:32:30.578554 15461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance:
uuid: "fba47c89bf8d4b6599b46a5f22ed7683"
format_stamp: "Formatted at 2026-09-26 16:32:30 on dist-test-slave-2zvq"
server_key: "9f5bd39a10c62a78408f9d74122e0767"
server_key_iv: "b72aa6fe120f12ca6c276c518e020ef0"
server_key_version: "encryptionkey@0"
I20260926 16:32:30.582114 15461 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260926 16:32:30.584524 15475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:30.585911 15461 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:32:30.586094 15461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-0/wal
uuid: "fba47c89bf8d4b6599b46a5f22ed7683"
format_stamp: "Formatted at 2026-09-26 16:32:30 on dist-test-slave-2zvq"
server_key: "9f5bd39a10c62a78408f9d74122e0767"
server_key_iv: "b72aa6fe120f12ca6c276c518e020ef0"
server_key_version: "encryptionkey@0"
I20260926 16:32:30.586230 15461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:30.602116 15461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:30.605419 15461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:30.605640 15461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:30.606251 15461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:32:30.607182 15461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:30.607254 15461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:30.607319 15461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:30.607369 15461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:30.616581 15461 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.193:35141
I20260926 16:32:30.616603 15588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.193:35141 every 8 connection(s)
I20260926 16:32:30.617226 15461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
I20260926 16:32:30.618070 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 15461
I20260926 16:32:30.618191 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance
I20260926 16:32:30.618489 11299 external_mini_cluster.cc:1428] Setting key b571f9b03aec00526aa5b75e38042d4d
Sep 26 16:32:30 dist-test-slave-2zvq krb5kdc[15355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440350, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.8.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.194@KRBTEST.COM" created.
I20260926 16:32:30.632342 15438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:30.619247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:34901 (local address 127.11.8.254:46615)
0926 16:32:30.619384 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:30.619389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:30.620717 (+ 1328us) server_negotiation.cc:407] Connection header received
0926 16:32:30.621664 (+ 947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:30.621667 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:30.621722 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:30.621799 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:30.622999 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:30.623556 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:30.624725 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:30.624978 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:30.625091 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:30.628151 (+ 3060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:30.628155 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:30.628157 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:30.628185 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:30.629705 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:30.630437 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:30.630440 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:30.630442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:30.630538 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:30.631029 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:30.631033 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:30.631035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:30.631305 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:30.631416 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:30.631942 (+ 526us) server_negotiation.cc:299] Negotiation successful
0926 16:32:30.632097 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 16:32:30.633396 15591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:30.619654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46615 (local address 127.11.8.193:34901)
0926 16:32:30.620547 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:30.620601 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:30.621456 (+ 855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:30.621984 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:30.621991 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:30.622362 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:30.622839 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:30.622852 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:30.623690 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:30.623694 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:30.624591 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:30.624600 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:30.625146 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:30.625150 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:30.625324 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:30.625996 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:30.626017 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:30.627954 (+ 1937us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:30.629892 (+ 1938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:30.629899 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:30.629922 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:30.630327 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:30.630667 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:30.630674 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:30.630679 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:30.630889 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:30.631418 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:30.631424 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:30.631682 (+ 258us) client_negotiation.cc:769] Sending connection context
0926 16:32:30.631844 (+ 162us) client_negotiation.cc:240] Negotiation successful
0926 16:32:30.632161 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":143,"threads_started":1}
I20260926 16:32:30.634939 15589 heartbeater.cc:347] Connected to a master server at 127.11.8.254:46615
I20260926 16:32:30.635213 15589 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:30.635699 15589 heartbeater.cc:511] Master 127.11.8.254:46615 requested a full tablet report, sending...
I20260926 16:32:30.637475 15399 ts_manager.cc:194] Registered new tserver with Master: fba47c89bf8d4b6599b46a5f22ed7683 (127.11.8.193:35141)
I20260926 16:32:30.639231 15399 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.193@KRBTEST.COM'} at 127.11.8.193:34901
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab.
Entry for principal kudu/127.11.8.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab.
I20260926 16:32:30.649729 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.194:0
--local_ip_for_outbound_sockets=127.11.8.194
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:46615
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:46813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:30.762220 15594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:30.762499 15594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:30.762542 15594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:30.767100 15594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:30.767150 15594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:30.767208 15594 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.194
I20260926 16:32:30.772228 15594 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:46813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:46615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15594
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.194
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:30.773164 15594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:30.774077 15594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:30.783083 15600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:30.783209 15602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:30.783083 15599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:30.783318 15594 server_base.cc:1034] running on GCE node
I20260926 16:32:30.783776 15594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:30.784438 15594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:30.785630 15594 hybrid_clock.cc:648] HybridClock initialized: now 1790440350785592 us; error 53 us; skew 500 ppm
Sep 26 16:32:30 dist-test-slave-2zvq krb5kdc[15355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440350, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:30.788568 15594 init.cc:377] Logged in from keytab as kudu/127.11.8.194@KRBTEST.COM (short username kudu)
I20260926 16:32:30.789407 15594 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:30.789507 15594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:30.789738 15594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:30.791889 15594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-1/data/instance:
uuid: "60766545a62b46fabbbdd35c758fcb6f"
format_stamp: "Formatted at 2026-09-26 16:32:30 on dist-test-slave-2zvq"
server_key: "e89b83a15c3c7cffba3826b644f4b1da"
server_key_iv: "b892a1c908ee495914a30c9e8b81372a"
server_key_version: "encryptionkey@0"
I20260926 16:32:30.792439 15594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance:
uuid: "60766545a62b46fabbbdd35c758fcb6f"
format_stamp: "Formatted at 2026-09-26 16:32:30 on dist-test-slave-2zvq"
server_key: "e89b83a15c3c7cffba3826b644f4b1da"
server_key_iv: "b892a1c908ee495914a30c9e8b81372a"
server_key_version: "encryptionkey@0"
I20260926 16:32:30.796079 15594 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260926 16:32:30.798384 15608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:30.799569 15594 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260926 16:32:30.799697 15594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-1/wal
uuid: "60766545a62b46fabbbdd35c758fcb6f"
format_stamp: "Formatted at 2026-09-26 16:32:30 on dist-test-slave-2zvq"
server_key: "e89b83a15c3c7cffba3826b644f4b1da"
server_key_iv: "b892a1c908ee495914a30c9e8b81372a"
server_key_version: "encryptionkey@0"
I20260926 16:32:30.799798 15594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:30.827862 15594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:30.831346 15594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:30.831549 15594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:30.832141 15594 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:32:30.832984 15594 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:30.833053 15594 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:30.833124 15594 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:30.833165 15594 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:30.841811 15594 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.194:36129
I20260926 16:32:30.841863 15721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.194:36129 every 8 connection(s)
I20260926 16:32:30.842480 15594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
I20260926 16:32:30.846104 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 15594
I20260926 16:32:30.846244 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance
I20260926 16:32:30.846555 11299 external_mini_cluster.cc:1428] Setting key c2b1a98b761656d590120c9c6ede9bf0
Sep 26 16:32:30 dist-test-slave-2zvq krb5kdc[15355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440350, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.8.195@KRBTEST.COM; defaulting to no policy
I20260926 16:32:30.855907 15438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:30.844178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:52001 (local address 127.11.8.254:46615)
0926 16:32:30.844319 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:30.844323 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:30.845487 (+ 1164us) server_negotiation.cc:407] Connection header received
0926 16:32:30.846459 (+ 972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:30.846462 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:30.846528 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:30.846628 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:30.847915 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:30.848434 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:30.849558 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:30.849781 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:30.849888 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:30.852261 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:30.852271 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:30.852274 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:30.852297 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:30.853750 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:30.854348 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:30.854351 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:30.854353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:30.854453 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:30.854807 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:30.854813 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:30.854818 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:30.855064 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:30.855168 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:30.855673 (+ 505us) server_negotiation.cc:299] Negotiation successful
0926 16:32:30.855792 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 16:32:30.856695 15724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:30.844564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46615 (local address 127.11.8.194:52001)
0926 16:32:30.845326 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:30.845366 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:30.846172 (+ 806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:30.846798 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:30.846804 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:30.847212 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:30.847766 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:30.847785 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:30.848567 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:30.848571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:30.849431 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:30.849440 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:30.849923 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:30.849927 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:30.850096 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:30.850667 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:30.850682 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:30.852103 (+ 1421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:30.853871 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:30.853876 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:30.853890 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:30.854209 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:30.854565 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:30.854567 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:30.854568 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:30.854692 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:30.855176 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:30.855181 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:30.855381 (+ 200us) client_negotiation.cc:769] Sending connection context
0926 16:32:30.855567 (+ 186us) client_negotiation.cc:240] Negotiation successful
0926 16:32:30.855798 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":108,"threads_started":1}
I20260926 16:32:30.857833 15722 heartbeater.cc:347] Connected to a master server at 127.11.8.254:46615
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.195@KRBTEST.COM" created.
I20260926 16:32:30.858095 15722 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:30.858578 15722 heartbeater.cc:511] Master 127.11.8.254:46615 requested a full tablet report, sending...
I20260926 16:32:30.859622 15399 ts_manager.cc:194] Registered new tserver with Master: 60766545a62b46fabbbdd35c758fcb6f (127.11.8.194:36129)
I20260926 16:32:30.860174 15399 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.194@KRBTEST.COM'} at 127.11.8.194:52001
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab.
Entry for principal kudu/127.11.8.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab.
I20260926 16:32:30.873878 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.195:0
--local_ip_for_outbound_sockets=127.11.8.195
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:46615
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:46813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:31.006649 15727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:31.006904 15727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:31.006948 15727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:31.010936 15727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:31.010985 15727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:31.011083 15727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.195
I20260926 16:32:31.016137 15727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:46813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:46615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.195
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:31.017145 15727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:31.018107 15727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 16:32:31.027508 15727 server_base.cc:1034] running on GCE node
W20260926 16:32:31.027472 15735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:31.027470 15733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:31.027551 15732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:31.028105 15727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:31.028821 15727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:31.030010 15727 hybrid_clock.cc:648] HybridClock initialized: now 1790440351029995 us; error 27 us; skew 500 ppm
Sep 26 16:32:31 dist-test-slave-2zvq krb5kdc[15355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440351, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:31.032960 15727 init.cc:377] Logged in from keytab as kudu/127.11.8.195@KRBTEST.COM (short username kudu)
I20260926 16:32:31.033818 15727 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:31.033921 15727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:31.034142 15727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:31.036770 15727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-2/data/instance:
uuid: "25a6fa1f02cd41cd840909438d481ca5"
format_stamp: "Formatted at 2026-09-26 16:32:31 on dist-test-slave-2zvq"
server_key: "880e5a621c5c259ef1129751230dcae9"
server_key_iv: "f791404102f6e34353bd4b7ec98f65bb"
server_key_version: "encryptionkey@0"
I20260926 16:32:31.037585 15727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance:
uuid: "25a6fa1f02cd41cd840909438d481ca5"
format_stamp: "Formatted at 2026-09-26 16:32:31 on dist-test-slave-2zvq"
server_key: "880e5a621c5c259ef1129751230dcae9"
server_key_iv: "f791404102f6e34353bd4b7ec98f65bb"
server_key_version: "encryptionkey@0"
I20260926 16:32:31.041817 15727 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 16:32:31.044538 15741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:31.045820 15727 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:32:31.045970 15727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-2/wal
uuid: "25a6fa1f02cd41cd840909438d481ca5"
format_stamp: "Formatted at 2026-09-26 16:32:31 on dist-test-slave-2zvq"
server_key: "880e5a621c5c259ef1129751230dcae9"
server_key_iv: "f791404102f6e34353bd4b7ec98f65bb"
server_key_version: "encryptionkey@0"
I20260926 16:32:31.046085 15727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:31.071673 15727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:31.074889 15727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:31.075115 15727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:31.075722 15727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:32:31.076555 15727 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:31.076634 15727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:31.076723 15727 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:31.076774 15727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:31.085778 15727 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.195:41505
I20260926 16:32:31.085788 15854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.195:41505 every 8 connection(s)
I20260926 16:32:31.086433 15727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
I20260926 16:32:31.094436 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 15727
I20260926 16:32:31.094561 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance
I20260926 16:32:31.094848 11299 external_mini_cluster.cc:1428] Setting key a224704836760fb4db38bd7b0927e0c3
Sep 26 16:32:31 dist-test-slave-2zvq krb5kdc[15355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440351, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:32:31.100023 15438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:31.088059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:44063 (local address 127.11.8.254:46615)
0926 16:32:31.088184 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:31.088195 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:31.089281 (+ 1086us) server_negotiation.cc:407] Connection header received
0926 16:32:31.090185 (+ 904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:31.090188 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:31.090243 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:31.090350 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:31.091478 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:31.092117 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:31.093247 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:31.093525 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:31.093616 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:31.096646 (+ 3030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:31.096649 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:31.096651 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:31.096675 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:31.098167 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:31.098714 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:31.098717 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:31.098718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:31.098807 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:31.099104 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:31.099106 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:31.099107 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:31.099279 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:31.099354 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:31.099794 (+ 440us) server_negotiation.cc:299] Negotiation successful
0926 16:32:31.099899 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 16:32:31.100826 15857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:31.088371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46615 (local address 127.11.8.195:44063)
0926 16:32:31.089111 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:31.089150 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:31.089984 (+ 834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:31.090540 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:31.090548 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:31.090880 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:31.091343 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:31.091356 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:31.092226 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:31.092228 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:31.093082 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:31.093091 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:31.093657 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:31.093660 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:31.093790 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:31.094558 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:31.094605 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:31.096483 (+ 1878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:31.098336 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:31.098342 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:31.098363 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:31.098626 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:31.098894 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:31.098896 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:31.098897 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:31.099026 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:31.099363 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:31.099368 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:31.099592 (+ 224us) client_negotiation.cc:769] Sending connection context
0926 16:32:31.099750 (+ 158us) client_negotiation.cc:240] Negotiation successful
0926 16:32:31.100003 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":106,"threads_started":1}
I20260926 16:32:31.102054 15855 heartbeater.cc:347] Connected to a master server at 127.11.8.254:46615
I20260926 16:32:31.102372 15855 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:31.102907 15855 heartbeater.cc:511] Master 127.11.8.254:46615 requested a full tablet report, sending...
I20260926 16:32:31.104061 15399 ts_manager.cc:194] Registered new tserver with Master: 25a6fa1f02cd41cd840909438d481ca5 (127.11.8.195:41505)
I20260926 16:32:31.104641 15399 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.195@KRBTEST.COM'} at 127.11.8.195:44063
I20260926 16:32:31.108834 11299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 16:32:31.118229 15438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:31.110825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59364 (local address 127.11.8.254:46615)
0926 16:32:31.110990 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:31.110997 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:31.111211 (+ 214us) server_negotiation.cc:407] Connection header received
0926 16:32:31.111362 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:31.111365 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:31.111415 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:31.111497 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:31.112190 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:31.112710 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:31.113546 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:31.113779 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:31.113892 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:31.115241 (+ 1349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:31.115245 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:31.115248 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:31.115273 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:31.116493 (+ 1220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:31.116901 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:31.116906 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:31.116908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:31.116998 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:31.117273 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:31.117277 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:31.117278 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:31.117542 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:31.117684 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:31.117913 (+ 229us) server_negotiation.cc:299] Negotiation successful
0926 16:32:31.118044 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 16:32:31.120870 15399 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59364:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:32:31.124001 15399 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:32:31.138012 15866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:31.132152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:36129 (local address 127.0.0.1:55312)
0926 16:32:31.132797 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:31.132845 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:31.132978 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:31.133613 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:31.133619 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:31.133641 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:31.133789 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:31.133797 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:31.135139 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:31.135144 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:31.136411 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:31.136422 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:31.137457 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:31.137461 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:31.137573 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:31.137618 (+ 45us) client_negotiation.cc:769] Sending connection context
0926 16:32:31.137718 (+ 100us) client_negotiation.cc:240] Negotiation successful
0926 16:32:31.137855 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"mutex_wait_us":27,"thread_start_us":104,"threads_started":1}
I20260926 16:32:31.138216 15867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:31.132354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55312 (local address 127.11.8.194:36129)
0926 16:32:31.132667 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:31.132679 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:31.133120 (+ 441us) server_negotiation.cc:407] Connection header received
0926 16:32:31.133203 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:31.133211 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:31.133450 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:31.133607 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:31.133927 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:31.135019 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:31.136557 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:31.137337 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:31.137445 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:31.137591 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:31.137957 (+ 366us) server_negotiation.cc:299] Negotiation successful
0926 16:32:31.138061 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":97,"threads_started":1}
I20260926 16:32:31.138162 15869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:31.132407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:35141 (local address 127.0.0.1:47988)
0926 16:32:31.133203 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:31.133214 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:31.133286 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:31.134145 (+ 859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:31.134149 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:31.134168 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:31.134256 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:31.134262 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:31.135675 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:31.135679 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:31.136803 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:31.136811 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:31.137757 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:31.137761 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:31.137849 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:31.137872 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 16:32:31.137947 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 16:32:31.138019 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":737,"mutex_wait_us":3,"thread_start_us":136,"threads_started":1}
I20260926 16:32:31.138672 15868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:31.132900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:41505 (local address 127.0.0.1:53058)
0926 16:32:31.133864 (+ 964us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:31.133878 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:31.133965 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:31.134527 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:31.134530 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:31.134548 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:31.134626 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:31.134631 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:31.135941 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:31.135947 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:31.137046 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:31.137056 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:31.138289 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:31.138294 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:31.138376 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:31.138397 (+ 21us) client_negotiation.cc:769] Sending connection context
0926 16:32:31.138472 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 16:32:31.138544 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":907,"mutex_wait_us":3,"thread_start_us":85,"threads_started":1}
I20260926 16:32:31.138933 15870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:31.132949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47988 (local address 127.11.8.193:35141)
0926 16:32:31.133525 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:31.133532 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:31.133555 (+ 23us) server_negotiation.cc:407] Connection header received
0926 16:32:31.133625 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:31.133630 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:31.133999 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:31.134145 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:31.134407 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:31.135555 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:31.136926 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:31.137631 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:31.137741 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:31.137870 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:31.138260 (+ 390us) server_negotiation.cc:299] Negotiation successful
0926 16:32:31.138380 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":114,"threads_started":1}
I20260926 16:32:31.139179 15871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:31.133021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53058 (local address 127.11.8.195:41505)
0926 16:32:31.133424 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:31.133443 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:31.134064 (+ 621us) server_negotiation.cc:407] Connection header received
0926 16:32:31.134143 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:31.134148 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:31.134387 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:31.134535 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:31.134742 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:31.135818 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:31.137170 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:31.137983 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:31.138074 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:31.138202 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:31.138925 (+ 723us) server_negotiation.cc:299] Negotiation successful
0926 16:32:31.139044 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":136,"threads_started":1}
I20260926 16:32:31.141064 15656 tablet_service.cc:1467] Processing CreateTablet for tablet 213136b35339401eaebcaddf8added3e (DEFAULT_TABLE table=test-table [id=b05d3803c1ab48bba3b49a9fc72ef363]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:31.141144 15523 tablet_service.cc:1467] Processing CreateTablet for tablet 213136b35339401eaebcaddf8added3e (DEFAULT_TABLE table=test-table [id=b05d3803c1ab48bba3b49a9fc72ef363]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:31.142091 15656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 213136b35339401eaebcaddf8added3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:31.142092 15523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 213136b35339401eaebcaddf8added3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:31.144218 15789 tablet_service.cc:1467] Processing CreateTablet for tablet 213136b35339401eaebcaddf8added3e (DEFAULT_TABLE table=test-table [id=b05d3803c1ab48bba3b49a9fc72ef363]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:31.145300 15789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 213136b35339401eaebcaddf8added3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:31.148061 15872 tablet_bootstrap.cc:492] T 213136b35339401eaebcaddf8added3e P fba47c89bf8d4b6599b46a5f22ed7683: Bootstrap starting.
I20260926 16:32:31.149704 15873 tablet_bootstrap.cc:492] T 213136b35339401eaebcaddf8added3e P 60766545a62b46fabbbdd35c758fcb6f: Bootstrap starting.
I20260926 16:32:31.151033 15872 tablet_bootstrap.cc:654] T 213136b35339401eaebcaddf8added3e P fba47c89bf8d4b6599b46a5f22ed7683: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:31.151754 15872 log.cc:824] T 213136b35339401eaebcaddf8added3e P fba47c89bf8d4b6599b46a5f22ed7683: Log is configured to *not* fsync() on all Append() calls
I20260926 16:32:31.151911 15873 tablet_bootstrap.cc:654] T 213136b35339401eaebcaddf8added3e P 60766545a62b46fabbbdd35c758fcb6f: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:31.152341 15874 tablet_bootstrap.cc:492] T 213136b35339401eaebcaddf8added3e P 25a6fa1f02cd41cd840909438d481ca5: Bootstrap starting.
I20260926 16:32:31.152956 15873 log.cc:824] T 213136b35339401eaebcaddf8added3e P 60766545a62b46fabbbdd35c758fcb6f: Log is configured to *not* fsync() on all Append() calls
I20260926 16:32:31.153831 15872 tablet_bootstrap.cc:492] T 213136b35339401eaebcaddf8added3e P fba47c89bf8d4b6599b46a5f22ed7683: No bootstrap required, opened a new log
I20260926 16:32:31.154018 15872 ts_tablet_manager.cc:1397] T 213136b35339401eaebcaddf8added3e P fba47c89bf8d4b6599b46a5f22ed7683: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 16:32:31.154778 15873 tablet_bootstrap.cc:492] T 213136b35339401eaebcaddf8added3e P 60766545a62b46fabbbdd35c758fcb6f: No bootstrap required, opened a new log
I20260926 16:32:31.154816 15874 tablet_bootstrap.cc:654] T 213136b35339401eaebcaddf8added3e P 25a6fa1f02cd41cd840909438d481ca5: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:31.154985 15873 ts_tablet_manager.cc:1397] T 213136b35339401eaebcaddf8added3e P 60766545a62b46fabbbdd35c758fcb6f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260926 16:32:31.155907 15874 log.cc:824] T 213136b35339401eaebcaddf8added3e P 25a6fa1f02cd41cd840909438d481ca5: Log is configured to *not* fsync() on all Append() calls
I20260926 16:32:31.157902 15874 tablet_bootstrap.cc:492] T 213136b35339401eaebcaddf8added3e P 25a6fa1f02cd41cd840909438d481ca5: No bootstrap required, opened a new log
I20260926 16:32:31.158075 15874 ts_tablet_manager.cc:1397] T 213136b35339401eaebcaddf8added3e P 25a6fa1f02cd41cd840909438d481ca5: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 16:32:31.158309 15873 raft_consensus.cc:348] T 213136b35339401eaebcaddf8added3e P 60766545a62b46fabbbdd35c758fcb6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60766545a62b46fabbbdd35c758fcb6f" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 36129 } } peers { permanent_uuid: "fba47c89bf8d4b6599b46a5f22ed7683" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 35141 } } peers { permanent_uuid: "25a6fa1f02cd41cd840909438d481ca5" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 41505 } }
I20260926 16:32:31.158582 15873 raft_consensus.cc:374] T 213136b35339401eaebcaddf8added3e P 60766545a62b46fabbbdd35c758fcb6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:31.158656 15873 raft_consensus.cc:729] T 213136b35339401eaebcaddf8added3e P 60766545a62b46fabbbdd35c758fcb6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60766545a62b46fabbbdd35c758fcb6f, State: Initialized, Role: FOLLOWER
I20260926 16:32:31.158675 15872 raft_consensus.cc:348] T 213136b35339401eaebcaddf8added3e P fba47c89bf8d4b6599b46a5f22ed7683 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60766545a62b46fabbbdd35c758fcb6f" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 36129 } } peers { permanent_uuid: "fba47c89bf8d4b6599b46a5f22ed7683" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 35141 } } peers { permanent_uuid: "25a6fa1f02cd41cd840909438d481ca5" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 41505 } }
I20260926 16:32:31.158974 15872 raft_consensus.cc:374] T 213136b35339401eaebcaddf8added3e P fba47c89bf8d4b6599b46a5f22ed7683 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:31.159035 15872 raft_consensus.cc:729] T 213136b35339401eaebcaddf8added3e P fba47c89bf8d4b6599b46a5f22ed7683 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fba47c89bf8d4b6599b46a5f22ed7683, State: Initialized, Role: FOLLOWER
I20260926 16:32:31.159134 15873 consensus_queue.cc:260] T 213136b35339401eaebcaddf8added3e P 60766545a62b46fabbbdd35c758fcb6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60766545a62b46fabbbdd35c758fcb6f" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 36129 } } peers { permanent_uuid: "fba47c89bf8d4b6599b46a5f22ed7683" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 35141 } } peers { permanent_uuid: "25a6fa1f02cd41cd840909438d481ca5" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 41505 } }
I20260926 16:32:31.159440 15872 consensus_queue.cc:260] T 213136b35339401eaebcaddf8added3e P fba47c89bf8d4b6599b46a5f22ed7683 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60766545a62b46fabbbdd35c758fcb6f" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 36129 } } peers { permanent_uuid: "fba47c89bf8d4b6599b46a5f22ed7683" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 35141 } } peers { permanent_uuid: "25a6fa1f02cd41cd840909438d481ca5" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 41505 } }
I20260926 16:32:31.159826 15722 heartbeater.cc:503] Master 127.11.8.254:46615 was elected leader, sending a full tablet report...
I20260926 16:32:31.160214 15873 ts_tablet_manager.cc:1428] T 213136b35339401eaebcaddf8added3e P 60766545a62b46fabbbdd35c758fcb6f: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260926 16:32:31.160430 15872 ts_tablet_manager.cc:1428] T 213136b35339401eaebcaddf8added3e P fba47c89bf8d4b6599b46a5f22ed7683: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260926 16:32:31.160774 15589 heartbeater.cc:503] Master 127.11.8.254:46615 was elected leader, sending a full tablet report...
I20260926 16:32:31.161710 15874 raft_consensus.cc:348] T 213136b35339401eaebcaddf8added3e P 25a6fa1f02cd41cd840909438d481ca5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60766545a62b46fabbbdd35c758fcb6f" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 36129 } } peers { permanent_uuid: "fba47c89bf8d4b6599b46a5f22ed7683" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 35141 } } peers { permanent_uuid: "25a6fa1f02cd41cd840909438d481ca5" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 41505 } }
I20260926 16:32:31.162038 15874 raft_consensus.cc:374] T 213136b35339401eaebcaddf8added3e P 25a6fa1f02cd41cd840909438d481ca5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:31.162133 15874 raft_consensus.cc:729] T 213136b35339401eaebcaddf8added3e P 25a6fa1f02cd41cd840909438d481ca5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25a6fa1f02cd41cd840909438d481ca5, State: Initialized, Role: FOLLOWER
I20260926 16:32:31.162571 15874 consensus_queue.cc:260] T 213136b35339401eaebcaddf8added3e P 25a6fa1f02cd41cd840909438d481ca5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60766545a62b46fabbbdd35c758fcb6f" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 36129 } } peers { permanent_uuid: "fba47c89bf8d4b6599b46a5f22ed7683" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 35141 } } peers { permanent_uuid: "25a6fa1f02cd41cd840909438d481ca5" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 41505 } }
I20260926 16:32:31.163223 15855 heartbeater.cc:503] Master 127.11.8.254:46615 was elected leader, sending a full tablet report...
I20260926 16:32:31.163542 15874 ts_tablet_manager.cc:1428] T 213136b35339401eaebcaddf8added3e P 25a6fa1f02cd41cd840909438d481ca5: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260926 16:32:31.315145 15880 raft_consensus.cc:482] T 213136b35339401eaebcaddf8added3e P 25a6fa1f02cd41cd840909438d481ca5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:32:31.315395 15880 raft_consensus.cc:504] T 213136b35339401eaebcaddf8added3e P 25a6fa1f02cd41cd840909438d481ca5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60766545a62b46fabbbdd35c758fcb6f" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 36129 } } peers { permanent_uuid: "fba47c89bf8d4b6599b46a5f22ed7683" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 35141 } } peers { permanent_uuid: "25a6fa1f02cd41cd840909438d481ca5" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 41505 } }
I20260926 16:32:31.316591 15880 leader_election.cc:288] T 213136b35339401eaebcaddf8added3e P 25a6fa1f02cd41cd840909438d481ca5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60766545a62b46fabbbdd35c758fcb6f (127.11.8.194:36129), fba47c89bf8d4b6599b46a5f22ed7683 (127.11.8.193:35141)
I20260926 16:32:31.321072 15857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:31.316914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:36129 (local address 127.11.8.195:50173)
0926 16:32:31.317094 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:31.317113 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:31.317249 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:31.317521 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:31.317524 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:31.317540 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:31.317692 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:31.317701 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:31.318738 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:31.318741 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:31.319804 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:31.319812 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:31.320639 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:31.320642 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:31.320719 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:31.320738 (+ 19us) client_negotiation.cc:769] Sending connection context
0926 16:32:31.320803 (+ 65us) client_negotiation.cc:240] Negotiation successful
0926 16:32:31.320900 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"mutex_wait_us":4}
I20260926 16:32:31.321069 15867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:31.317145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:50173 (local address 127.11.8.194:36129)
0926 16:32:31.317292 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:31.317297 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:31.317314 (+ 17us) server_negotiation.cc:407] Connection header received
0926 16:32:31.317374 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:31.317376 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:31.317417 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:31.317498 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:31.317814 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:31.318637 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:31.319934 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:31.320519 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:31.320638 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:31.320690 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:31.320847 (+ 157us) server_negotiation.cc:299] Negotiation successful
0926 16:32:31.320933 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 16:32:31.321353 15881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:31.316914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:35141 (local address 127.11.8.195:43737)
0926 16:32:31.317522 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:31.317535 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:31.317655 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:31.317959 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:31.317961 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:31.317976 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:31.318061 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:31.318065 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:31.318986 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:31.318990 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:31.320036 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:31.320046 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:31.320943 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:31.320946 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:31.321032 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:31.321052 (+ 20us) client_negotiation.cc:769] Sending connection context
0926 16:32:31.321130 (+ 78us) client_negotiation.cc:240] Negotiation successful
0926 16:32:31.321194 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"mutex_wait_us":282,"thread_start_us":145,"threads_started":1}
I20260926 16:32:31.321398 15870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:31.317056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:43737 (local address 127.11.8.193:35141)
0926 16:32:31.317223 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:31.317228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:31.317615 (+ 387us) server_negotiation.cc:407] Connection header received
0926 16:32:31.317792 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:31.317813 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:31.317867 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:31.317942 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:31.318161 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:31.318869 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:31.320153 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:31.320800 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:31.320916 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:31.320960 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:31.321184 (+ 224us) server_negotiation.cc:299] Negotiation successful
0926 16:32:31.321273 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 16:32:31.321694 15676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "213136b35339401eaebcaddf8added3e" candidate_uuid: "25a6fa1f02cd41cd840909438d481ca5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60766545a62b46fabbbdd35c758fcb6f" is_pre_election: true
I20260926 16:32:31.322038 15676 raft_consensus.cc:2454] T 213136b35339401eaebcaddf8added3e P 60766545a62b46fabbbdd35c758fcb6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25a6fa1f02cd41cd840909438d481ca5 in term 0.
I20260926 16:32:31.322084 15543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "213136b35339401eaebcaddf8added3e" candidate_uuid: "25a6fa1f02cd41cd840909438d481ca5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fba47c89bf8d4b6599b46a5f22ed7683" is_pre_election: true
I20260926 16:32:31.322373 15543 raft_consensus.cc:2454] T 213136b35339401eaebcaddf8added3e P fba47c89bf8d4b6599b46a5f22ed7683 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25a6fa1f02cd41cd840909438d481ca5 in term 0.
I20260926 16:32:31.322619 15743 leader_election.cc:302] T 213136b35339401eaebcaddf8added3e P 25a6fa1f02cd41cd840909438d481ca5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25a6fa1f02cd41cd840909438d481ca5, 60766545a62b46fabbbdd35c758fcb6f; no voters:
I20260926 16:32:31.322917 15880 raft_consensus.cc:2781] T 213136b35339401eaebcaddf8added3e P 25a6fa1f02cd41cd840909438d481ca5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:32:31.322995 15880 raft_consensus.cc:482] T 213136b35339401eaebcaddf8added3e P 25a6fa1f02cd41cd840909438d481ca5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:32:31.323025 15880 raft_consensus.cc:3037] T 213136b35339401eaebcaddf8added3e P 25a6fa1f02cd41cd840909438d481ca5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:31.324146 15880 raft_consensus.cc:504] T 213136b35339401eaebcaddf8added3e P 25a6fa1f02cd41cd840909438d481ca5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60766545a62b46fabbbdd35c758fcb6f" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 36129 } } peers { permanent_uuid: "fba47c89bf8d4b6599b46a5f22ed7683" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 35141 } } peers { permanent_uuid: "25a6fa1f02cd41cd840909438d481ca5" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 41505 } }
I20260926 16:32:31.324504 15880 leader_election.cc:288] T 213136b35339401eaebcaddf8added3e P 25a6fa1f02cd41cd840909438d481ca5 [CANDIDATE]: Term 1 election: Requested vote from peers 60766545a62b46fabbbdd35c758fcb6f (127.11.8.194:36129), fba47c89bf8d4b6599b46a5f22ed7683 (127.11.8.193:35141)
I20260926 16:32:31.324852 15676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "213136b35339401eaebcaddf8added3e" candidate_uuid: "25a6fa1f02cd41cd840909438d481ca5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60766545a62b46fabbbdd35c758fcb6f"
I20260926 16:32:31.324887 15543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "213136b35339401eaebcaddf8added3e" candidate_uuid: "25a6fa1f02cd41cd840909438d481ca5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fba47c89bf8d4b6599b46a5f22ed7683"
I20260926 16:32:31.324996 15543 raft_consensus.cc:3037] T 213136b35339401eaebcaddf8added3e P fba47c89bf8d4b6599b46a5f22ed7683 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:31.324987 15676 raft_consensus.cc:3037] T 213136b35339401eaebcaddf8added3e P 60766545a62b46fabbbdd35c758fcb6f [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:31.326364 15676 raft_consensus.cc:2454] T 213136b35339401eaebcaddf8added3e P 60766545a62b46fabbbdd35c758fcb6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25a6fa1f02cd41cd840909438d481ca5 in term 1.
I20260926 16:32:31.326364 15543 raft_consensus.cc:2454] T 213136b35339401eaebcaddf8added3e P fba47c89bf8d4b6599b46a5f22ed7683 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25a6fa1f02cd41cd840909438d481ca5 in term 1.
I20260926 16:32:31.326650 15743 leader_election.cc:302] T 213136b35339401eaebcaddf8added3e P 25a6fa1f02cd41cd840909438d481ca5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25a6fa1f02cd41cd840909438d481ca5, 60766545a62b46fabbbdd35c758fcb6f; no voters:
I20260926 16:32:31.326807 15880 raft_consensus.cc:2781] T 213136b35339401eaebcaddf8added3e P 25a6fa1f02cd41cd840909438d481ca5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:32:31.327044 15880 raft_consensus.cc:686] T 213136b35339401eaebcaddf8added3e P 25a6fa1f02cd41cd840909438d481ca5 [term 1 LEADER]: Becoming Leader. State: Replica: 25a6fa1f02cd41cd840909438d481ca5, State: Running, Role: LEADER
I20260926 16:32:31.327368 15880 consensus_queue.cc:237] T 213136b35339401eaebcaddf8added3e P 25a6fa1f02cd41cd840909438d481ca5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60766545a62b46fabbbdd35c758fcb6f" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 36129 } } peers { permanent_uuid: "fba47c89bf8d4b6599b46a5f22ed7683" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 35141 } } peers { permanent_uuid: "25a6fa1f02cd41cd840909438d481ca5" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 41505 } }
I20260926 16:32:31.330436 15399 catalog_manager.cc:5495] T 213136b35339401eaebcaddf8added3e P 25a6fa1f02cd41cd840909438d481ca5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 25a6fa1f02cd41cd840909438d481ca5 (127.11.8.195). New cstate: current_term: 1 leader_uuid: "25a6fa1f02cd41cd840909438d481ca5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60766545a62b46fabbbdd35c758fcb6f" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 36129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fba47c89bf8d4b6599b46a5f22ed7683" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 35141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25a6fa1f02cd41cd840909438d481ca5" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 41505 } health_report { overall_health: HEALTHY } } }
W20260926 16:32:31.337245 15856 tablet.cc:2369] T 213136b35339401eaebcaddf8added3e P 25a6fa1f02cd41cd840909438d481ca5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 16:32:31.343521 15723 tablet.cc:2369] T 213136b35339401eaebcaddf8added3e P 60766545a62b46fabbbdd35c758fcb6f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 16:32:31.368458 15590 tablet.cc:2369] T 213136b35339401eaebcaddf8added3e P fba47c89bf8d4b6599b46a5f22ed7683: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:32:31.373358 15871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:31.370016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53074 (local address 127.11.8.195:41505)
0926 16:32:31.370158 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:31.370162 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:31.370176 (+ 14us) server_negotiation.cc:407] Connection header received
0926 16:32:31.370285 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:31.370287 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:31.370378 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:31.370467 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:32:31.370692 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:31.371211 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:31.372044 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:31.372249 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:31.372347 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:31.372591 (+ 244us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:32:31.372998 (+ 407us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:32:31.373091 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:31.373163 (+ 72us) server_negotiation.cc:299] Negotiation successful
0926 16:32:31.373216 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 16:32:31.379061 15543 raft_consensus.cc:1261] T 213136b35339401eaebcaddf8added3e P fba47c89bf8d4b6599b46a5f22ed7683 [term 1 FOLLOWER]: Refusing update from remote peer 25a6fa1f02cd41cd840909438d481ca5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:32:31.379061 15676 raft_consensus.cc:1261] T 213136b35339401eaebcaddf8added3e P 60766545a62b46fabbbdd35c758fcb6f [term 1 FOLLOWER]: Refusing update from remote peer 25a6fa1f02cd41cd840909438d481ca5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:32:31.379899 15882 consensus_queue.cc:1035] T 213136b35339401eaebcaddf8added3e P 25a6fa1f02cd41cd840909438d481ca5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fba47c89bf8d4b6599b46a5f22ed7683" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 35141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:32:31.380244 15880 consensus_queue.cc:1035] T 213136b35339401eaebcaddf8added3e P 25a6fa1f02cd41cd840909438d481ca5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60766545a62b46fabbbdd35c758fcb6f" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 36129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:32:31.387420 15887 mvcc.cc:205] Tried to move back new op lower bound from 7333643679240810496 to 7333643679039729664. Current Snapshot: MvccSnapshot[applied={T|T < 7333643679240810496}]
I20260926 16:32:31.389729 15886 mvcc.cc:205] Tried to move back new op lower bound from 7333643679240810496 to 7333643679039729664. Current Snapshot: MvccSnapshot[applied={T|T < 7333643679240810496}]
I20260926 16:32:31.395424 15397 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:32:31.395619 15397 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:32:31.398115 15397 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fc3725660bc94165966a5e5a51d5a59f: Sending DeleteTablet for 3 replicas of tablet 213136b35339401eaebcaddf8added3e
I20260926 16:32:31.398911 15523 tablet_service.cc:1514] Processing DeleteTablet for tablet 213136b35339401eaebcaddf8added3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:32:31 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:47988
I20260926 16:32:31.398993 15656 tablet_service.cc:1514] Processing DeleteTablet for tablet 213136b35339401eaebcaddf8added3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:32:31 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:55312
I20260926 16:32:31.399019 15789 tablet_service.cc:1514] Processing DeleteTablet for tablet 213136b35339401eaebcaddf8added3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:32:31 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:53058
I20260926 16:32:31.399514 15895 tablet_replica.cc:330] T 213136b35339401eaebcaddf8added3e P fba47c89bf8d4b6599b46a5f22ed7683: stopping tablet replica
I20260926 16:32:31.399557 15897 tablet_replica.cc:330] T 213136b35339401eaebcaddf8added3e P 25a6fa1f02cd41cd840909438d481ca5: stopping tablet replica
I20260926 16:32:31.399843 15897 raft_consensus.cc:2229] T 213136b35339401eaebcaddf8added3e P 25a6fa1f02cd41cd840909438d481ca5 [term 1 LEADER]: Raft consensus shutting down.
I20260926 16:32:31.399844 15895 raft_consensus.cc:2229] T 213136b35339401eaebcaddf8added3e P fba47c89bf8d4b6599b46a5f22ed7683 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:32:31.400167 15895 raft_consensus.cc:2258] T 213136b35339401eaebcaddf8added3e P fba47c89bf8d4b6599b46a5f22ed7683 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:32:31.400189 15897 raft_consensus.cc:2258] T 213136b35339401eaebcaddf8added3e P 25a6fa1f02cd41cd840909438d481ca5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:32:31.400328 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 15461
I20260926 16:32:31.400502 15896 tablet_replica.cc:330] T 213136b35339401eaebcaddf8added3e P 60766545a62b46fabbbdd35c758fcb6f: stopping tablet replica
I20260926 16:32:31.400722 15896 raft_consensus.cc:2229] T 213136b35339401eaebcaddf8added3e P 60766545a62b46fabbbdd35c758fcb6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:32:31.400979 15896 raft_consensus.cc:2258] T 213136b35339401eaebcaddf8added3e P 60766545a62b46fabbbdd35c758fcb6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:32:31.402535 15897 ts_tablet_manager.cc:1905] T 213136b35339401eaebcaddf8added3e P 25a6fa1f02cd41cd840909438d481ca5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:32:31.404951 15897 ts_tablet_manager.cc:1918] T 213136b35339401eaebcaddf8added3e P 25a6fa1f02cd41cd840909438d481ca5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:32:31.405061 15897 log.cc:1196] T 213136b35339401eaebcaddf8added3e P 25a6fa1f02cd41cd840909438d481ca5: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790440324823152-11299-0/minicluster-data/ts-2/wal/wals/213136b35339401eaebcaddf8added3e
I20260926 16:32:31.405355 15897 ts_tablet_manager.cc:1939] T 213136b35339401eaebcaddf8added3e P 25a6fa1f02cd41cd840909438d481ca5: Deleting consensus metadata
I20260926 16:32:31.406430 15384 catalog_manager.cc:4841] TS 25a6fa1f02cd41cd840909438d481ca5 (127.11.8.195:41505): tablet 213136b35339401eaebcaddf8added3e (table test-table [id=b05d3803c1ab48bba3b49a9fc72ef363]) successfully deleted
I20260926 16:32:31.407250 15896 ts_tablet_manager.cc:1905] T 213136b35339401eaebcaddf8added3e P 60766545a62b46fabbbdd35c758fcb6f: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 16:32:31.408026 15385 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.8.193:35141 (error 108)
I20260926 16:32:31.408843 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 15594
I20260926 16:32:31.409181 15868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:31.408760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:35141 (local address 127.0.0.1:47992)
0926 16:32:31.408987 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:31.409080 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.193:35141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
W20260926 16:32:31.409621 15385 catalog_manager.cc:4584] TS fba47c89bf8d4b6599b46a5f22ed7683 (127.11.8.193:35141): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 213136b35339401eaebcaddf8added3e: Network error: Client connection negotiation failed: client connection to 127.11.8.193:35141: connect: Connection refused (error 111)
I20260926 16:32:31.415946 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 15727
I20260926 16:32:31.416626 15868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:31.416291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:36129 (local address 127.0.0.1:55328)
0926 16:32:31.416466 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:31.416528 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.194:36129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260926 16:32:31.421547 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 15368
2026-09-26T16:32:31Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2413 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-26T16:32:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:32:31Z Disabled control of system clock
I20260926 16:32:31.451521 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:35997
--webserver_interface=127.11.8.254
--webserver_port=0
--builtin_ntp_servers=127.11.8.212:43621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:35997
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260926 16:32:31.559049 15902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:31.559299 15902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:31.559337 15902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:31.563253 15902 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:32:31.563304 15902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:31.563320 15902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:31.563336 15902 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:32:31.563352 15902 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:32:31.567900 15902 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:43621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:35997
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:35997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:31.568832 15902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:31.570123 15902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:31.578645 15907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:31.578701 15910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:31.578701 15908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:31.579069 15902 server_base.cc:1034] running on GCE node
I20260926 16:32:31.579607 15902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:31.580693 15902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:31.581872 15902 hybrid_clock.cc:648] HybridClock initialized: now 1790440351581856 us; error 27 us; skew 500 ppm
I20260926 16:32:31.583853 15902 webserver.cc:460] Webserver started at http://127.11.8.254:46309/ using document root <none> and password file <none>
I20260926 16:32:31.584465 15902 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:31.584528 15902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:31.584754 15902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:31.586988 15902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "696fdcaff34d4c728db0f138817cc6ce"
format_stamp: "Formatted at 2026-09-26 16:32:31 on dist-test-slave-2zvq"
server_key: "44566aa72fd61f047b3be421b4ce1429"
server_key_iv: "53353bf231f424a89c58e88377ee9ff9"
server_key_version: "encryptionkey@0"
I20260926 16:32:31.587594 15902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "696fdcaff34d4c728db0f138817cc6ce"
format_stamp: "Formatted at 2026-09-26 16:32:31 on dist-test-slave-2zvq"
server_key: "44566aa72fd61f047b3be421b4ce1429"
server_key_iv: "53353bf231f424a89c58e88377ee9ff9"
server_key_version: "encryptionkey@0"
I20260926 16:32:31.591483 15902 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260926 16:32:31.593947 15916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:31.595306 15902 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260926 16:32:31.595453 15902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "696fdcaff34d4c728db0f138817cc6ce"
format_stamp: "Formatted at 2026-09-26 16:32:31 on dist-test-slave-2zvq"
server_key: "44566aa72fd61f047b3be421b4ce1429"
server_key_iv: "53353bf231f424a89c58e88377ee9ff9"
server_key_version: "encryptionkey@0"
I20260926 16:32:31.595577 15902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:31.600174 15902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:31.600781 15902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:31.600968 15902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:31.607815 15902 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:35997
I20260926 16:32:31.607841 15968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:35997 every 8 connection(s)
I20260926 16:32:31.608704 15902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:32:31.611877 15969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:31.616590 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 15902
I20260926 16:32:31.616714 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:32:31.617010 11299 external_mini_cluster.cc:1428] Setting key 6e7c408d05fc352e5111ce0b9ee43e03
I20260926 16:32:31.617792 15969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 696fdcaff34d4c728db0f138817cc6ce: Bootstrap starting.
I20260926 16:32:31.620174 15969 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 696fdcaff34d4c728db0f138817cc6ce: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:31.621009 15969 log.cc:824] T 00000000000000000000000000000000 P 696fdcaff34d4c728db0f138817cc6ce: Log is configured to *not* fsync() on all Append() calls
I20260926 16:32:31.622681 15972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:31.618450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50062 (local address 127.11.8.254:35997)
0926 16:32:31.619080 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:31.619095 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:31.619129 (+ 34us) server_negotiation.cc:407] Connection header received
0926 16:32:31.619741 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:31.619758 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:31.620112 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:31.620485 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:31.621261 (+ 776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:31.621268 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:31.621285 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:32:31.621295 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:31.621349 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:32:31.621386 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:31.621500 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:31.621650 (+ 150us) server_negotiation.cc:299] Negotiation successful
0926 16:32:31.621901 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":118,"threads_started":1}
I20260926 16:32:31.623167 15969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 696fdcaff34d4c728db0f138817cc6ce: No bootstrap required, opened a new log
I20260926 16:32:31.626992 15969 raft_consensus.cc:348] T 00000000000000000000000000000000 P 696fdcaff34d4c728db0f138817cc6ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "696fdcaff34d4c728db0f138817cc6ce" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 35997 } }
I20260926 16:32:31.627305 15969 raft_consensus.cc:374] T 00000000000000000000000000000000 P 696fdcaff34d4c728db0f138817cc6ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:31.627383 15969 raft_consensus.cc:729] T 00000000000000000000000000000000 P 696fdcaff34d4c728db0f138817cc6ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 696fdcaff34d4c728db0f138817cc6ce, State: Initialized, Role: FOLLOWER
I20260926 16:32:31.627866 15969 consensus_queue.cc:260] T 00000000000000000000000000000000 P 696fdcaff34d4c728db0f138817cc6ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "696fdcaff34d4c728db0f138817cc6ce" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 35997 } }
I20260926 16:32:31.628017 15969 raft_consensus.cc:388] T 00000000000000000000000000000000 P 696fdcaff34d4c728db0f138817cc6ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:32:31.628103 15969 raft_consensus.cc:482] T 00000000000000000000000000000000 P 696fdcaff34d4c728db0f138817cc6ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:32:31.628202 15969 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 696fdcaff34d4c728db0f138817cc6ce [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:31.629217 15969 raft_consensus.cc:504] T 00000000000000000000000000000000 P 696fdcaff34d4c728db0f138817cc6ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "696fdcaff34d4c728db0f138817cc6ce" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 35997 } }
I20260926 16:32:31.629565 15969 leader_election.cc:302] T 00000000000000000000000000000000 P 696fdcaff34d4c728db0f138817cc6ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 696fdcaff34d4c728db0f138817cc6ce; no voters:
I20260926 16:32:31.629876 15969 leader_election.cc:288] T 00000000000000000000000000000000 P 696fdcaff34d4c728db0f138817cc6ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:32:31.630036 15974 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 696fdcaff34d4c728db0f138817cc6ce [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:32:31.630294 15974 raft_consensus.cc:686] T 00000000000000000000000000000000 P 696fdcaff34d4c728db0f138817cc6ce [term 1 LEADER]: Becoming Leader. State: Replica: 696fdcaff34d4c728db0f138817cc6ce, State: Running, Role: LEADER
I20260926 16:32:31.630599 15974 consensus_queue.cc:237] T 00000000000000000000000000000000 P 696fdcaff34d4c728db0f138817cc6ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "696fdcaff34d4c728db0f138817cc6ce" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 35997 } }
I20260926 16:32:31.631263 15969 sys_catalog.cc:564] T 00000000000000000000000000000000 P 696fdcaff34d4c728db0f138817cc6ce [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:32:31.632544 15975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 696fdcaff34d4c728db0f138817cc6ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "696fdcaff34d4c728db0f138817cc6ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "696fdcaff34d4c728db0f138817cc6ce" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 35997 } } }
I20260926 16:32:31.632548 15976 sys_catalog.cc:455] T 00000000000000000000000000000000 P 696fdcaff34d4c728db0f138817cc6ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 696fdcaff34d4c728db0f138817cc6ce. Latest consensus state: current_term: 1 leader_uuid: "696fdcaff34d4c728db0f138817cc6ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "696fdcaff34d4c728db0f138817cc6ce" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 35997 } } }
I20260926 16:32:31.632699 15976 sys_catalog.cc:458] T 00000000000000000000000000000000 P 696fdcaff34d4c728db0f138817cc6ce [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:31.632699 15975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 696fdcaff34d4c728db0f138817cc6ce [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:31.633178 15984 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:32:31.636091 15984 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:32:31.641067 15984 catalog_manager.cc:1347] Generated new cluster ID: 5d7566ab5fef4910a6e4e2c0a7162711
I20260926 16:32:31.641129 15984 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:32:31.658543 15984 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:32:31.658905 15984 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:32:31.665899 15984 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 696fdcaff34d4c728db0f138817cc6ce: Generated new TSK 0
I20260926 16:32:31.666707 15984 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:32:31.670531 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.193:0
--local_ip_for_outbound_sockets=127.11.8.193
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:35997
--builtin_ntp_servers=127.11.8.212:43621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260926 16:32:31.780429 15993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:31.780668 15993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:31.780761 15993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:31.784686 15993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:31.784737 15993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:31.784852 15993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.193
I20260926 16:32:31.789747 15993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:43621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:35997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.193
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:31.790884 15993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:31.792265 15993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:31.801679 16001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:31.801748 15998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:31.801780 15993 server_base.cc:1034] running on GCE node
W20260926 16:32:31.801676 15999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:31.802345 15993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:31.802943 15993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:31.804133 15993 hybrid_clock.cc:648] HybridClock initialized: now 1790440351804121 us; error 24 us; skew 500 ppm
I20260926 16:32:31.805899 15993 webserver.cc:460] Webserver started at http://127.11.8.193:44719/ using document root <none> and password file <none>
I20260926 16:32:31.806533 15993 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:31.806627 15993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:31.806849 15993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:31.808784 15993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-0/data/instance:
uuid: "20e00debb88546768ea83f687bc4c23e"
format_stamp: "Formatted at 2026-09-26 16:32:31 on dist-test-slave-2zvq"
server_key: "35e730458e981f89629b322af2030ee7"
server_key_iv: "a90ec01bf703a5cd8dcb71977d59d385"
server_key_version: "encryptionkey@0"
I20260926 16:32:31.809340 15993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance:
uuid: "20e00debb88546768ea83f687bc4c23e"
format_stamp: "Formatted at 2026-09-26 16:32:31 on dist-test-slave-2zvq"
server_key: "35e730458e981f89629b322af2030ee7"
server_key_iv: "a90ec01bf703a5cd8dcb71977d59d385"
server_key_version: "encryptionkey@0"
I20260926 16:32:31.812824 15993 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260926 16:32:31.815320 16007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:31.816470 15993 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:32:31.816604 15993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-0/wal
uuid: "20e00debb88546768ea83f687bc4c23e"
format_stamp: "Formatted at 2026-09-26 16:32:31 on dist-test-slave-2zvq"
server_key: "35e730458e981f89629b322af2030ee7"
server_key_iv: "a90ec01bf703a5cd8dcb71977d59d385"
server_key_version: "encryptionkey@0"
I20260926 16:32:31.816716 15993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:31.821291 15993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:31.821870 15993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:31.822059 15993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:31.822674 15993 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:32:31.823536 15993 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:31.823582 15993 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:31.823618 15993 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:31.823639 15993 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:31.832114 15993 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.193:42205
I20260926 16:32:31.832140 16120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.193:42205 every 8 connection(s)
I20260926 16:32:31.833205 15993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
I20260926 16:32:31.836535 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 15993
I20260926 16:32:31.836668 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance
I20260926 16:32:31.836988 11299 external_mini_cluster.cc:1428] Setting key 1fcd1a6fa4b235a348b11800d82924cd
I20260926 16:32:31.839344 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.194:0
--local_ip_for_outbound_sockets=127.11.8.194
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:35997
--builtin_ntp_servers=127.11.8.212:43621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260926 16:32:31.839799 15972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:31.835038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:59597 (local address 127.11.8.254:35997)
0926 16:32:31.835176 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:31.835181 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:31.836264 (+ 1083us) server_negotiation.cc:407] Connection header received
0926 16:32:31.837930 (+ 1666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:31.837933 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:31.837971 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:31.838040 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:31.839321 (+ 1281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:31.839324 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:31.839326 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:32:31.839337 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:31.839357 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:32:31.839365 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:31.839491 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:31.839659 (+ 168us) server_negotiation.cc:299] Negotiation successful
0926 16:32:31.839697 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 16:32:31.840947 16123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:31.835437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:35997 (local address 127.11.8.193:59597)
0926 16:32:31.836088 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:31.836129 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:31.837714 (+ 1585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:31.838238 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:31.838250 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:31.838386 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:31.838978 (+ 592us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:32:31.838983 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:31.839044 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:32:31.839137 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:31.839489 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:31.839502 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:31.839512 (+ 10us) client_negotiation.cc:769] Sending connection context
0926 16:32:31.839693 (+ 181us) client_negotiation.cc:240] Negotiation successful
0926 16:32:31.839946 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":110,"threads_started":1}
I20260926 16:32:31.842224 16121 heartbeater.cc:347] Connected to a master server at 127.11.8.254:35997
I20260926 16:32:31.842468 16121 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:31.842885 16121 heartbeater.cc:511] Master 127.11.8.254:35997 requested a full tablet report, sending...
I20260926 16:32:31.844370 15933 ts_manager.cc:194] Registered new tserver with Master: 20e00debb88546768ea83f687bc4c23e (127.11.8.193:42205)
W20260926 16:32:31.947697 16124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:31.947981 16124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:31.948024 16124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:31.952126 16124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:31.952183 16124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:31.952241 16124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.194
I20260926 16:32:31.957044 16124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:43621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:35997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.194
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:31.958114 16124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:31.959574 16124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:31.968981 16130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:31.969048 16132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:31.969099 16124 server_base.cc:1034] running on GCE node
W20260926 16:32:31.968961 16129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:31.969640 16124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:31.970239 16124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:31.971438 16124 hybrid_clock.cc:648] HybridClock initialized: now 1790440351971423 us; error 27 us; skew 500 ppm
I20260926 16:32:31.973134 16124 webserver.cc:460] Webserver started at http://127.11.8.194:36779/ using document root <none> and password file <none>
I20260926 16:32:31.973693 16124 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:31.973753 16124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:31.973991 16124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:31.976028 16124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-1/data/instance:
uuid: "c9822f9ab29e4925b5e0a33c0590230a"
format_stamp: "Formatted at 2026-09-26 16:32:31 on dist-test-slave-2zvq"
server_key: "30ca2e9ec35dc97d0b665b393aa1d513"
server_key_iv: "cb72b2cd4dba0c90c461912792a6d0c4"
server_key_version: "encryptionkey@0"
I20260926 16:32:31.976603 16124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance:
uuid: "c9822f9ab29e4925b5e0a33c0590230a"
format_stamp: "Formatted at 2026-09-26 16:32:31 on dist-test-slave-2zvq"
server_key: "30ca2e9ec35dc97d0b665b393aa1d513"
server_key_iv: "cb72b2cd4dba0c90c461912792a6d0c4"
server_key_version: "encryptionkey@0"
I20260926 16:32:31.980139 16124 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 16:32:31.982604 16138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:31.983775 16124 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:32:31.983914 16124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-1/wal
uuid: "c9822f9ab29e4925b5e0a33c0590230a"
format_stamp: "Formatted at 2026-09-26 16:32:31 on dist-test-slave-2zvq"
server_key: "30ca2e9ec35dc97d0b665b393aa1d513"
server_key_iv: "cb72b2cd4dba0c90c461912792a6d0c4"
server_key_version: "encryptionkey@0"
I20260926 16:32:31.984026 16124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:31.988663 16124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:31.989289 16124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:31.989485 16124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:31.990034 16124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:32:31.990995 16124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:31.991041 16124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:31.991080 16124 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:31.991137 16124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:32.000056 16124 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.194:45845
I20260926 16:32:32.000087 16251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.194:45845 every 8 connection(s)
I20260926 16:32:32.000924 16124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
I20260926 16:32:32.005421 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 16124
I20260926 16:32:32.005518 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance
I20260926 16:32:32.005838 11299 external_mini_cluster.cc:1428] Setting key 1ae004b4e977e357214c7113108bff39
I20260926 16:32:32.007227 15972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:32.002679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:46259 (local address 127.11.8.254:35997)
0926 16:32:32.002814 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:32.002818 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:32.003886 (+ 1068us) server_negotiation.cc:407] Connection header received
0926 16:32:32.005232 (+ 1346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:32.005235 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:32.005281 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:32.005360 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:32.006683 (+ 1323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:32.006689 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:32.006691 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:32:32.006703 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:32.006726 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:32:32.006736 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:32.006831 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:32.007049 (+ 218us) server_negotiation.cc:299] Negotiation successful
0926 16:32:32.007095 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 16:32:32.008068 16254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:32.003047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:35997 (local address 127.11.8.194:46259)
0926 16:32:32.003722 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:32.003762 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:32.004984 (+ 1222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:32.005555 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:32.005583 (+ 28us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:32.005717 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:32.006384 (+ 667us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:32:32.006389 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:32.006446 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:32:32.006541 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:32.006877 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:32.006888 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:32.006894 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 16:32:32.007070 (+ 176us) client_negotiation.cc:240] Negotiation successful
0926 16:32:32.007305 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":106,"threads_started":1}
I20260926 16:32:32.008085 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.195:0
--local_ip_for_outbound_sockets=127.11.8.195
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:35997
--builtin_ntp_servers=127.11.8.212:43621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260926 16:32:32.009183 16252 heartbeater.cc:347] Connected to a master server at 127.11.8.254:35997
I20260926 16:32:32.009402 16252 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:32.009793 16252 heartbeater.cc:511] Master 127.11.8.254:35997 requested a full tablet report, sending...
I20260926 16:32:32.010836 15933 ts_manager.cc:194] Registered new tserver with Master: c9822f9ab29e4925b5e0a33c0590230a (127.11.8.194:45845)
W20260926 16:32:32.120960 16255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:32.121196 16255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:32.121233 16255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:32.125448 16255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:32.125501 16255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:32.125555 16255 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.195
I20260926 16:32:32.130113 16255 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:43621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:35997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16255
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.195
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:32.131136 16255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:32.132441 16255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:32.141522 16261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:32.141476 16263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:32.141470 16260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:32.141917 16255 server_base.cc:1034] running on GCE node
I20260926 16:32:32.142349 16255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:32.143011 16255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:32.144181 16255 hybrid_clock.cc:648] HybridClock initialized: now 1790440352144159 us; error 31 us; skew 500 ppm
I20260926 16:32:32.145957 16255 webserver.cc:460] Webserver started at http://127.11.8.195:35773/ using document root <none> and password file <none>
I20260926 16:32:32.146567 16255 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:32.146631 16255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:32.146799 16255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:32.148829 16255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-2/data/instance:
uuid: "1b372973aa7542b8866230b37b9da2df"
format_stamp: "Formatted at 2026-09-26 16:32:32 on dist-test-slave-2zvq"
server_key: "6171e183eba8627f7cc7e16de30422dd"
server_key_iv: "3855f083b7ee7176afb01b7a3412a03f"
server_key_version: "encryptionkey@0"
I20260926 16:32:32.149334 16255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance:
uuid: "1b372973aa7542b8866230b37b9da2df"
format_stamp: "Formatted at 2026-09-26 16:32:32 on dist-test-slave-2zvq"
server_key: "6171e183eba8627f7cc7e16de30422dd"
server_key_iv: "3855f083b7ee7176afb01b7a3412a03f"
server_key_version: "encryptionkey@0"
I20260926 16:32:32.152899 16255 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260926 16:32:32.155278 16269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:32.156430 16255 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 16:32:32.156575 16255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-2/wal
uuid: "1b372973aa7542b8866230b37b9da2df"
format_stamp: "Formatted at 2026-09-26 16:32:32 on dist-test-slave-2zvq"
server_key: "6171e183eba8627f7cc7e16de30422dd"
server_key_iv: "3855f083b7ee7176afb01b7a3412a03f"
server_key_version: "encryptionkey@0"
I20260926 16:32:32.156683 16255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:32.161088 16255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:32.161664 16255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:32.161842 16255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:32.162405 16255 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:32:32.163249 16255 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:32.163316 16255 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:32.163376 16255 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:32.163429 16255 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:32.172297 16255 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.195:33499
I20260926 16:32:32.172324 16382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.195:33499 every 8 connection(s)
I20260926 16:32:32.173116 16255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
I20260926 16:32:32.174167 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 16255
I20260926 16:32:32.174309 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance
I20260926 16:32:32.174599 11299 external_mini_cluster.cc:1428] Setting key 4b5bcba9c182485556edcb47c92e08f7
I20260926 16:32:32.179322 15972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:32.174954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:37725 (local address 127.11.8.254:35997)
0926 16:32:32.175121 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:32.175126 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:32.176344 (+ 1218us) server_negotiation.cc:407] Connection header received
0926 16:32:32.177691 (+ 1347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:32.177694 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:32.177735 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:32.177810 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:32.178853 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:32.178857 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:32.178859 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:32:32.178871 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:32.178892 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:32:32.178902 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:32.179000 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:32.179156 (+ 156us) server_negotiation.cc:299] Negotiation successful
0926 16:32:32.179198 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 16:32:32.180106 16385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:32.175522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:35997 (local address 127.11.8.195:37725)
0926 16:32:32.176215 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:32.176252 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:32.177495 (+ 1243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:32.177976 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:32.177982 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:32.178069 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:32.178605 (+ 536us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:32:32.178607 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:32.178647 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:32:32.178713 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:32.178996 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:32.179005 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:32.179012 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 16:32:32.179187 (+ 175us) client_negotiation.cc:240] Negotiation successful
0926 16:32:32.179414 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":152,"threads_started":1}
I20260926 16:32:32.181088 16383 heartbeater.cc:347] Connected to a master server at 127.11.8.254:35997
I20260926 16:32:32.181301 16383 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:32.181677 16383 heartbeater.cc:511] Master 127.11.8.254:35997 requested a full tablet report, sending...
I20260926 16:32:32.182621 15933 ts_manager.cc:194] Registered new tserver with Master: 1b372973aa7542b8866230b37b9da2df (127.11.8.195:33499)
I20260926 16:32:32.188275 11299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 16:32:32.191696 15972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:32.190158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50074 (local address 127.11.8.254:35997)
0926 16:32:32.190340 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:32.190343 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:32.190442 (+ 99us) server_negotiation.cc:407] Connection header received
0926 16:32:32.190544 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:32.190546 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:32.190578 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:32.190639 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:32.191393 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:32.191396 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:32.191398 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:32:32.191414 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:32.191429 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:32:32.191436 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:32.191503 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:32.191568 (+ 65us) server_negotiation.cc:299] Negotiation successful
0926 16:32:32.191598 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 16:32:32.194449 15933 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50074:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:32:32.197458 15933 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:32:32.208947 16397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:32.206425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52658 (local address 127.11.8.194:45845)
0926 16:32:32.207207 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:32.207219 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:32.207242 (+ 23us) server_negotiation.cc:407] Connection header received
0926 16:32:32.207326 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:32.207332 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:32.207495 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:32.207630 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:32.208443 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:32.208448 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:32.208457 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:32:32.208467 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:32.208502 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:32:32.208555 (+ 53us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:32.208657 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:32.208732 (+ 75us) server_negotiation.cc:299] Negotiation successful
0926 16:32:32.208797 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":679,"thread_start_us":96,"threads_started":1}
I20260926 16:32:32.208937 16394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:32.205442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:33499 (local address 127.0.0.1:55944)
0926 16:32:32.205811 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:32.205862 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:32.205967 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:32.207155 (+ 1188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:32.207163 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:32.207248 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:32.207890 (+ 642us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:32:32.207894 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:32.207916 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:32:32.208000 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:32.208566 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:32.208578 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:32.208587 (+ 9us) client_negotiation.cc:769] Sending connection context
0926 16:32:32.208710 (+ 123us) client_negotiation.cc:240] Negotiation successful
0926 16:32:32.208777 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":70,"threads_started":1}
I20260926 16:32:32.209232 16399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:32.206666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43472 (local address 127.11.8.193:42205)
0926 16:32:32.207275 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:32.207286 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:32.207304 (+ 18us) server_negotiation.cc:407] Connection header received
0926 16:32:32.207349 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:32.207353 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:32.207495 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:32.207630 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:32.208611 (+ 981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:32.208617 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:32.208628 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:32:32.208641 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:32.208681 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:32:32.208711 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:32.208798 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:32.209010 (+ 212us) server_negotiation.cc:299] Negotiation successful
0926 16:32:32.209072 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":169,"threads_started":1}
I20260926 16:32:32.209474 16396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:32.206518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:42205 (local address 127.0.0.1:43472)
0926 16:32:32.207063 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:32.207076 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:32.207142 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:32.207749 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:32.207753 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:32.207769 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:32.208439 (+ 670us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:32:32.208442 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:32.208451 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:32:32.208474 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:32.208838 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:32.208841 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:32.208844 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 16:32:32.208910 (+ 66us) client_negotiation.cc:240] Negotiation successful
0926 16:32:32.208937 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":85,"threads_started":1}
I20260926 16:32:32.209517 16398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:32.206191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:45845 (local address 127.0.0.1:52658)
0926 16:32:32.206819 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:32.206832 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:32.206902 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:32.207640 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:32.207644 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:32.207659 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:32.208150 (+ 491us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:32:32.208152 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:32.208163 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:32:32.208187 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:32.208632 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:32.208636 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:32.208639 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 16:32:32.208722 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 16:32:32.208784 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"mutex_wait_us":1,"thread_start_us":107,"threads_started":1}
I20260926 16:32:32.209920 16395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:32.205592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55944 (local address 127.11.8.195:33499)
0926 16:32:32.205933 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:32.205945 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:32.205980 (+ 35us) server_negotiation.cc:407] Connection header received
0926 16:32:32.206056 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:32.206065 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:32.206296 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:32.206443 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:32.208321 (+ 1878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:32.208327 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:32.208348 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:32:32.208358 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:32.208420 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:32:32.208452 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:32.208576 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:32.208870 (+ 294us) server_negotiation.cc:299] Negotiation successful
0926 16:32:32.208940 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":134,"threads_started":1}
I20260926 16:32:32.212203 16317 tablet_service.cc:1467] Processing CreateTablet for tablet 3cb32f24117247b4a58d7699faeec398 (DEFAULT_TABLE table=test-table [id=fed09a97f9c94ca5bb99fb2910119e75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:32.212371 16186 tablet_service.cc:1467] Processing CreateTablet for tablet 3cb32f24117247b4a58d7699faeec398 (DEFAULT_TABLE table=test-table [id=fed09a97f9c94ca5bb99fb2910119e75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:32.212371 16055 tablet_service.cc:1467] Processing CreateTablet for tablet 3cb32f24117247b4a58d7699faeec398 (DEFAULT_TABLE table=test-table [id=fed09a97f9c94ca5bb99fb2910119e75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:32.213238 16317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3cb32f24117247b4a58d7699faeec398. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:32.213327 16055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3cb32f24117247b4a58d7699faeec398. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:32.213328 16186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3cb32f24117247b4a58d7699faeec398. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:32.219440 16400 tablet_bootstrap.cc:492] T 3cb32f24117247b4a58d7699faeec398 P 1b372973aa7542b8866230b37b9da2df: Bootstrap starting.
I20260926 16:32:32.221057 16401 tablet_bootstrap.cc:492] T 3cb32f24117247b4a58d7699faeec398 P 20e00debb88546768ea83f687bc4c23e: Bootstrap starting.
I20260926 16:32:32.221206 16402 tablet_bootstrap.cc:492] T 3cb32f24117247b4a58d7699faeec398 P c9822f9ab29e4925b5e0a33c0590230a: Bootstrap starting.
I20260926 16:32:32.222571 16400 tablet_bootstrap.cc:654] T 3cb32f24117247b4a58d7699faeec398 P 1b372973aa7542b8866230b37b9da2df: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:32.223026 16401 tablet_bootstrap.cc:654] T 3cb32f24117247b4a58d7699faeec398 P 20e00debb88546768ea83f687bc4c23e: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:32.223425 16400 log.cc:824] T 3cb32f24117247b4a58d7699faeec398 P 1b372973aa7542b8866230b37b9da2df: Log is configured to *not* fsync() on all Append() calls
I20260926 16:32:32.223661 16402 tablet_bootstrap.cc:654] T 3cb32f24117247b4a58d7699faeec398 P c9822f9ab29e4925b5e0a33c0590230a: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:32.223845 16401 log.cc:824] T 3cb32f24117247b4a58d7699faeec398 P 20e00debb88546768ea83f687bc4c23e: Log is configured to *not* fsync() on all Append() calls
I20260926 16:32:32.224421 16402 log.cc:824] T 3cb32f24117247b4a58d7699faeec398 P c9822f9ab29e4925b5e0a33c0590230a: Log is configured to *not* fsync() on all Append() calls
I20260926 16:32:32.225555 16400 tablet_bootstrap.cc:492] T 3cb32f24117247b4a58d7699faeec398 P 1b372973aa7542b8866230b37b9da2df: No bootstrap required, opened a new log
I20260926 16:32:32.225751 16400 ts_tablet_manager.cc:1397] T 3cb32f24117247b4a58d7699faeec398 P 1b372973aa7542b8866230b37b9da2df: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 16:32:32.225778 16401 tablet_bootstrap.cc:492] T 3cb32f24117247b4a58d7699faeec398 P 20e00debb88546768ea83f687bc4c23e: No bootstrap required, opened a new log
I20260926 16:32:32.225934 16401 ts_tablet_manager.cc:1397] T 3cb32f24117247b4a58d7699faeec398 P 20e00debb88546768ea83f687bc4c23e: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260926 16:32:32.226145 16402 tablet_bootstrap.cc:492] T 3cb32f24117247b4a58d7699faeec398 P c9822f9ab29e4925b5e0a33c0590230a: No bootstrap required, opened a new log
I20260926 16:32:32.226348 16402 ts_tablet_manager.cc:1397] T 3cb32f24117247b4a58d7699faeec398 P c9822f9ab29e4925b5e0a33c0590230a: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260926 16:32:32.229943 16402 raft_consensus.cc:348] T 3cb32f24117247b4a58d7699faeec398 P c9822f9ab29e4925b5e0a33c0590230a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b372973aa7542b8866230b37b9da2df" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33499 } } peers { permanent_uuid: "c9822f9ab29e4925b5e0a33c0590230a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45845 } } peers { permanent_uuid: "20e00debb88546768ea83f687bc4c23e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42205 } }
I20260926 16:32:32.230211 16402 raft_consensus.cc:374] T 3cb32f24117247b4a58d7699faeec398 P c9822f9ab29e4925b5e0a33c0590230a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:32.230319 16402 raft_consensus.cc:729] T 3cb32f24117247b4a58d7699faeec398 P c9822f9ab29e4925b5e0a33c0590230a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9822f9ab29e4925b5e0a33c0590230a, State: Initialized, Role: FOLLOWER
I20260926 16:32:32.230425 16401 raft_consensus.cc:348] T 3cb32f24117247b4a58d7699faeec398 P 20e00debb88546768ea83f687bc4c23e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b372973aa7542b8866230b37b9da2df" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33499 } } peers { permanent_uuid: "c9822f9ab29e4925b5e0a33c0590230a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45845 } } peers { permanent_uuid: "20e00debb88546768ea83f687bc4c23e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42205 } }
I20260926 16:32:32.230425 16400 raft_consensus.cc:348] T 3cb32f24117247b4a58d7699faeec398 P 1b372973aa7542b8866230b37b9da2df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b372973aa7542b8866230b37b9da2df" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33499 } } peers { permanent_uuid: "c9822f9ab29e4925b5e0a33c0590230a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45845 } } peers { permanent_uuid: "20e00debb88546768ea83f687bc4c23e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42205 } }
I20260926 16:32:32.230685 16400 raft_consensus.cc:374] T 3cb32f24117247b4a58d7699faeec398 P 1b372973aa7542b8866230b37b9da2df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:32.230685 16401 raft_consensus.cc:374] T 3cb32f24117247b4a58d7699faeec398 P 20e00debb88546768ea83f687bc4c23e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:32.230744 16401 raft_consensus.cc:729] T 3cb32f24117247b4a58d7699faeec398 P 20e00debb88546768ea83f687bc4c23e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20e00debb88546768ea83f687bc4c23e, State: Initialized, Role: FOLLOWER
I20260926 16:32:32.230786 16400 raft_consensus.cc:729] T 3cb32f24117247b4a58d7699faeec398 P 1b372973aa7542b8866230b37b9da2df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b372973aa7542b8866230b37b9da2df, State: Initialized, Role: FOLLOWER
I20260926 16:32:32.230822 16402 consensus_queue.cc:260] T 3cb32f24117247b4a58d7699faeec398 P c9822f9ab29e4925b5e0a33c0590230a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b372973aa7542b8866230b37b9da2df" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33499 } } peers { permanent_uuid: "c9822f9ab29e4925b5e0a33c0590230a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45845 } } peers { permanent_uuid: "20e00debb88546768ea83f687bc4c23e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42205 } }
I20260926 16:32:32.231222 16400 consensus_queue.cc:260] T 3cb32f24117247b4a58d7699faeec398 P 1b372973aa7542b8866230b37b9da2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b372973aa7542b8866230b37b9da2df" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33499 } } peers { permanent_uuid: "c9822f9ab29e4925b5e0a33c0590230a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45845 } } peers { permanent_uuid: "20e00debb88546768ea83f687bc4c23e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42205 } }
I20260926 16:32:32.231222 16401 consensus_queue.cc:260] T 3cb32f24117247b4a58d7699faeec398 P 20e00debb88546768ea83f687bc4c23e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b372973aa7542b8866230b37b9da2df" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33499 } } peers { permanent_uuid: "c9822f9ab29e4925b5e0a33c0590230a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45845 } } peers { permanent_uuid: "20e00debb88546768ea83f687bc4c23e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42205 } }
I20260926 16:32:32.231616 16252 heartbeater.cc:503] Master 127.11.8.254:35997 was elected leader, sending a full tablet report...
I20260926 16:32:32.232000 16402 ts_tablet_manager.cc:1428] T 3cb32f24117247b4a58d7699faeec398 P c9822f9ab29e4925b5e0a33c0590230a: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260926 16:32:32.232242 16400 ts_tablet_manager.cc:1428] T 3cb32f24117247b4a58d7699faeec398 P 1b372973aa7542b8866230b37b9da2df: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260926 16:32:32.232242 16401 ts_tablet_manager.cc:1428] T 3cb32f24117247b4a58d7699faeec398 P 20e00debb88546768ea83f687bc4c23e: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260926 16:32:32.232535 16383 heartbeater.cc:503] Master 127.11.8.254:35997 was elected leader, sending a full tablet report...
I20260926 16:32:32.232589 16121 heartbeater.cc:503] Master 127.11.8.254:35997 was elected leader, sending a full tablet report...
W20260926 16:32:32.251740 16253 tablet.cc:2369] T 3cb32f24117247b4a58d7699faeec398 P c9822f9ab29e4925b5e0a33c0590230a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 16:32:32.334528 16122 tablet.cc:2369] T 3cb32f24117247b4a58d7699faeec398 P 20e00debb88546768ea83f687bc4c23e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 16:32:32.424060 16384 tablet.cc:2369] T 3cb32f24117247b4a58d7699faeec398 P 1b372973aa7542b8866230b37b9da2df: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:32:32.536267 16408 raft_consensus.cc:482] T 3cb32f24117247b4a58d7699faeec398 P 20e00debb88546768ea83f687bc4c23e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:32:32.536504 16408 raft_consensus.cc:504] T 3cb32f24117247b4a58d7699faeec398 P 20e00debb88546768ea83f687bc4c23e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b372973aa7542b8866230b37b9da2df" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33499 } } peers { permanent_uuid: "c9822f9ab29e4925b5e0a33c0590230a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45845 } } peers { permanent_uuid: "20e00debb88546768ea83f687bc4c23e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42205 } }
I20260926 16:32:32.537652 16408 leader_election.cc:288] T 3cb32f24117247b4a58d7699faeec398 P 20e00debb88546768ea83f687bc4c23e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b372973aa7542b8866230b37b9da2df (127.11.8.195:33499), c9822f9ab29e4925b5e0a33c0590230a (127.11.8.194:45845)
I20260926 16:32:32.540155 16395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:32.538053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:56231 (local address 127.11.8.195:33499)
0926 16:32:32.538206 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:32.538210 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:32.538368 (+ 158us) server_negotiation.cc:407] Connection header received
0926 16:32:32.538736 (+ 368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:32.538742 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:32.538800 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:32.538885 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:32.539683 (+ 798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:32.539688 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:32.539690 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:32:32.539702 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:32.539723 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:32:32.539733 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:32.539822 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:32.539932 (+ 110us) server_negotiation.cc:299] Negotiation successful
0926 16:32:32.540010 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 16:32:32.540158 16409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:32.537922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:33499 (local address 127.11.8.193:56231)
0926 16:32:32.538226 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:32.538243 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:32.538460 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:32.538921 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:32.538924 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:32.538962 (+ 38us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:32.539486 (+ 524us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:32:32.539489 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:32.539502 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:32:32.539536 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:32.539830 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:32.539834 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:32.539838 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 16:32:32.539925 (+ 87us) client_negotiation.cc:240] Negotiation successful
0926 16:32:32.539993 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":92,"threads_started":1}
I20260926 16:32:32.540256 16397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:32.538493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:42597 (local address 127.11.8.194:45845)
0926 16:32:32.538610 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:32.538617 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:32.538777 (+ 160us) server_negotiation.cc:407] Connection header received
0926 16:32:32.538912 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:32.538915 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:32.538953 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:32.539040 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:32.539863 (+ 823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:32.539869 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:32.539875 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:32:32.539891 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:32.539913 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:32:32.539923 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:32.540007 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:32.540095 (+ 88us) server_negotiation.cc:299] Negotiation successful
0926 16:32:32.540130 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 16:32:32.540324 16410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:32.538377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:45845 (local address 127.11.8.193:42597)
0926 16:32:32.538680 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:32.538696 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:32.538796 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:32.539064 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:32.539066 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:32.539078 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:32.539681 (+ 603us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:32:32.539683 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:32.539694 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:32:32.539719 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:32.540011 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:32.540014 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:32.540016 (+ 2us) client_negotiation.cc:769] Sending connection context
0926 16:32:32.540099 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 16:32:32.540131 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":83,"threads_started":1}
I20260926 16:32:32.541088 16206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cb32f24117247b4a58d7699faeec398" candidate_uuid: "20e00debb88546768ea83f687bc4c23e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9822f9ab29e4925b5e0a33c0590230a" is_pre_election: true
I20260926 16:32:32.541096 16337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cb32f24117247b4a58d7699faeec398" candidate_uuid: "20e00debb88546768ea83f687bc4c23e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b372973aa7542b8866230b37b9da2df" is_pre_election: true
I20260926 16:32:32.541373 16337 raft_consensus.cc:2454] T 3cb32f24117247b4a58d7699faeec398 P 1b372973aa7542b8866230b37b9da2df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20e00debb88546768ea83f687bc4c23e in term 0.
I20260926 16:32:32.541380 16206 raft_consensus.cc:2454] T 3cb32f24117247b4a58d7699faeec398 P c9822f9ab29e4925b5e0a33c0590230a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20e00debb88546768ea83f687bc4c23e in term 0.
I20260926 16:32:32.541824 16011 leader_election.cc:302] T 3cb32f24117247b4a58d7699faeec398 P 20e00debb88546768ea83f687bc4c23e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b372973aa7542b8866230b37b9da2df, 20e00debb88546768ea83f687bc4c23e; no voters:
I20260926 16:32:32.542069 16408 raft_consensus.cc:2781] T 3cb32f24117247b4a58d7699faeec398 P 20e00debb88546768ea83f687bc4c23e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:32:32.542141 16408 raft_consensus.cc:482] T 3cb32f24117247b4a58d7699faeec398 P 20e00debb88546768ea83f687bc4c23e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:32:32.542171 16408 raft_consensus.cc:3037] T 3cb32f24117247b4a58d7699faeec398 P 20e00debb88546768ea83f687bc4c23e [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:32.543224 16408 raft_consensus.cc:504] T 3cb32f24117247b4a58d7699faeec398 P 20e00debb88546768ea83f687bc4c23e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b372973aa7542b8866230b37b9da2df" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33499 } } peers { permanent_uuid: "c9822f9ab29e4925b5e0a33c0590230a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45845 } } peers { permanent_uuid: "20e00debb88546768ea83f687bc4c23e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42205 } }
I20260926 16:32:32.543543 16408 leader_election.cc:288] T 3cb32f24117247b4a58d7699faeec398 P 20e00debb88546768ea83f687bc4c23e [CANDIDATE]: Term 1 election: Requested vote from peers 1b372973aa7542b8866230b37b9da2df (127.11.8.195:33499), c9822f9ab29e4925b5e0a33c0590230a (127.11.8.194:45845)
I20260926 16:32:32.543927 16337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cb32f24117247b4a58d7699faeec398" candidate_uuid: "20e00debb88546768ea83f687bc4c23e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b372973aa7542b8866230b37b9da2df"
I20260926 16:32:32.543974 16206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cb32f24117247b4a58d7699faeec398" candidate_uuid: "20e00debb88546768ea83f687bc4c23e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9822f9ab29e4925b5e0a33c0590230a"
I20260926 16:32:32.544036 16337 raft_consensus.cc:3037] T 3cb32f24117247b4a58d7699faeec398 P 1b372973aa7542b8866230b37b9da2df [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:32.544109 16206 raft_consensus.cc:3037] T 3cb32f24117247b4a58d7699faeec398 P c9822f9ab29e4925b5e0a33c0590230a [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:32.545050 16337 raft_consensus.cc:2454] T 3cb32f24117247b4a58d7699faeec398 P 1b372973aa7542b8866230b37b9da2df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20e00debb88546768ea83f687bc4c23e in term 1.
I20260926 16:32:32.545243 16206 raft_consensus.cc:2454] T 3cb32f24117247b4a58d7699faeec398 P c9822f9ab29e4925b5e0a33c0590230a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20e00debb88546768ea83f687bc4c23e in term 1.
I20260926 16:32:32.545313 16011 leader_election.cc:302] T 3cb32f24117247b4a58d7699faeec398 P 20e00debb88546768ea83f687bc4c23e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b372973aa7542b8866230b37b9da2df, 20e00debb88546768ea83f687bc4c23e; no voters:
I20260926 16:32:32.545598 16408 raft_consensus.cc:2781] T 3cb32f24117247b4a58d7699faeec398 P 20e00debb88546768ea83f687bc4c23e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:32:32.545814 16408 raft_consensus.cc:686] T 3cb32f24117247b4a58d7699faeec398 P 20e00debb88546768ea83f687bc4c23e [term 1 LEADER]: Becoming Leader. State: Replica: 20e00debb88546768ea83f687bc4c23e, State: Running, Role: LEADER
I20260926 16:32:32.546142 16408 consensus_queue.cc:237] T 3cb32f24117247b4a58d7699faeec398 P 20e00debb88546768ea83f687bc4c23e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b372973aa7542b8866230b37b9da2df" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33499 } } peers { permanent_uuid: "c9822f9ab29e4925b5e0a33c0590230a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45845 } } peers { permanent_uuid: "20e00debb88546768ea83f687bc4c23e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42205 } }
I20260926 16:32:32.549319 15932 catalog_manager.cc:5495] T 3cb32f24117247b4a58d7699faeec398 P 20e00debb88546768ea83f687bc4c23e reported cstate change: term changed from 0 to 1, leader changed from <none> to 20e00debb88546768ea83f687bc4c23e (127.11.8.193). New cstate: current_term: 1 leader_uuid: "20e00debb88546768ea83f687bc4c23e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b372973aa7542b8866230b37b9da2df" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9822f9ab29e4925b5e0a33c0590230a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20e00debb88546768ea83f687bc4c23e" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42205 } health_report { overall_health: HEALTHY } } }
I20260926 16:32:32.570065 16399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:32.568669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43476 (local address 127.11.8.193:42205)
0926 16:32:32.568797 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:32.568800 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:32.568814 (+ 14us) server_negotiation.cc:407] Connection header received
0926 16:32:32.568951 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:32.568953 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:32.569011 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:32.569079 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:32.569696 (+ 617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:32.569699 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:32.569701 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:32:32.569711 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:32.569729 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:32:32.569737 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:32.569806 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:32.569863 (+ 57us) server_negotiation.cc:299] Negotiation successful
0926 16:32:32.569894 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 16:32:32.575856 16206 raft_consensus.cc:1261] T 3cb32f24117247b4a58d7699faeec398 P c9822f9ab29e4925b5e0a33c0590230a [term 1 FOLLOWER]: Refusing update from remote peer 20e00debb88546768ea83f687bc4c23e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:32:32.575846 16337 raft_consensus.cc:1261] T 3cb32f24117247b4a58d7699faeec398 P 1b372973aa7542b8866230b37b9da2df [term 1 FOLLOWER]: Refusing update from remote peer 20e00debb88546768ea83f687bc4c23e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:32:32.576598 16411 consensus_queue.cc:1035] T 3cb32f24117247b4a58d7699faeec398 P 20e00debb88546768ea83f687bc4c23e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b372973aa7542b8866230b37b9da2df" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:32:32.576766 16408 consensus_queue.cc:1035] T 3cb32f24117247b4a58d7699faeec398 P 20e00debb88546768ea83f687bc4c23e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9822f9ab29e4925b5e0a33c0590230a" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:32:32.595794 15932 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50074:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:32:32.596015 15932 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50074:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:32:32.598924 15932 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 696fdcaff34d4c728db0f138817cc6ce: Sending DeleteTablet for 3 replicas of tablet 3cb32f24117247b4a58d7699faeec398
I20260926 16:32:32.599825 16317 tablet_service.cc:1514] Processing DeleteTablet for tablet 3cb32f24117247b4a58d7699faeec398 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:32:32 UTC) from {username='slave'} at 127.0.0.1:55944
I20260926 16:32:32.599937 16055 tablet_service.cc:1514] Processing DeleteTablet for tablet 3cb32f24117247b4a58d7699faeec398 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:32:32 UTC) from {username='slave'} at 127.0.0.1:43472
I20260926 16:32:32.600049 16186 tablet_service.cc:1514] Processing DeleteTablet for tablet 3cb32f24117247b4a58d7699faeec398 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:32:32 UTC) from {username='slave'} at 127.0.0.1:52658
I20260926 16:32:32.600484 16424 tablet_replica.cc:330] T 3cb32f24117247b4a58d7699faeec398 P 1b372973aa7542b8866230b37b9da2df: stopping tablet replica
I20260926 16:32:32.600520 16425 tablet_replica.cc:330] T 3cb32f24117247b4a58d7699faeec398 P 20e00debb88546768ea83f687bc4c23e: stopping tablet replica
I20260926 16:32:32.600608 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 15993
I20260926 16:32:32.600832 16425 raft_consensus.cc:2229] T 3cb32f24117247b4a58d7699faeec398 P 20e00debb88546768ea83f687bc4c23e [term 1 LEADER]: Raft consensus shutting down.
I20260926 16:32:32.600854 16424 raft_consensus.cc:2229] T 3cb32f24117247b4a58d7699faeec398 P 1b372973aa7542b8866230b37b9da2df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:32:32.601262 16424 raft_consensus.cc:2258] T 3cb32f24117247b4a58d7699faeec398 P 1b372973aa7542b8866230b37b9da2df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:32:32.601291 16426 tablet_replica.cc:330] T 3cb32f24117247b4a58d7699faeec398 P c9822f9ab29e4925b5e0a33c0590230a: stopping tablet replica
I20260926 16:32:32.601542 16426 raft_consensus.cc:2229] T 3cb32f24117247b4a58d7699faeec398 P c9822f9ab29e4925b5e0a33c0590230a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:32:32.601836 16426 raft_consensus.cc:2258] T 3cb32f24117247b4a58d7699faeec398 P c9822f9ab29e4925b5e0a33c0590230a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:32:32.602919 16424 ts_tablet_manager.cc:1905] T 3cb32f24117247b4a58d7699faeec398 P 1b372973aa7542b8866230b37b9da2df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:32:32.603588 16426 ts_tablet_manager.cc:1905] T 3cb32f24117247b4a58d7699faeec398 P c9822f9ab29e4925b5e0a33c0590230a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:32:32.605268 16424 ts_tablet_manager.cc:1918] T 3cb32f24117247b4a58d7699faeec398 P 1b372973aa7542b8866230b37b9da2df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:32:32.605389 16424 log.cc:1196] T 3cb32f24117247b4a58d7699faeec398 P 1b372973aa7542b8866230b37b9da2df: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-2/wal/wals/3cb32f24117247b4a58d7699faeec398
I20260926 16:32:32.605683 16424 ts_tablet_manager.cc:1939] T 3cb32f24117247b4a58d7699faeec398 P 1b372973aa7542b8866230b37b9da2df: Deleting consensus metadata
I20260926 16:32:32.606864 15920 catalog_manager.cc:4841] TS 1b372973aa7542b8866230b37b9da2df (127.11.8.195:33499): tablet 3cb32f24117247b4a58d7699faeec398 (table test-table [id=fed09a97f9c94ca5bb99fb2910119e75]) successfully deleted
I20260926 16:32:32.606911 16426 ts_tablet_manager.cc:1918] T 3cb32f24117247b4a58d7699faeec398 P c9822f9ab29e4925b5e0a33c0590230a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:32:32.607013 16426 log.cc:1196] T 3cb32f24117247b4a58d7699faeec398 P c9822f9ab29e4925b5e0a33c0590230a: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790440324823152-11299-0/minicluster-data/ts-1/wal/wals/3cb32f24117247b4a58d7699faeec398
I20260926 16:32:32.607336 16426 ts_tablet_manager.cc:1939] T 3cb32f24117247b4a58d7699faeec398 P c9822f9ab29e4925b5e0a33c0590230a: Deleting consensus metadata
I20260926 16:32:32.608553 15920 catalog_manager.cc:4841] TS c9822f9ab29e4925b5e0a33c0590230a (127.11.8.194:45845): tablet 3cb32f24117247b4a58d7699faeec398 (table test-table [id=fed09a97f9c94ca5bb99fb2910119e75]) successfully deleted
W20260926 16:32:32.609210 15920 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.8.193:42205 (error 108)
I20260926 16:32:32.610001 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 16124
I20260926 16:32:32.610210 16398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:32.609917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:42205 (local address 127.0.0.1:43478)
0926 16:32:32.610047 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:32.610126 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.193:42205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260926 16:32:32.610961 15920 catalog_manager.cc:4584] TS 20e00debb88546768ea83f687bc4c23e (127.11.8.193:42205): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3cb32f24117247b4a58d7699faeec398: Network error: Client connection negotiation failed: client connection to 127.11.8.193:42205: connect: Connection refused (error 111)
I20260926 16:32:32.615295 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 16255
I20260926 16:32:32.620877 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 15902
2026-09-26T16:32:32Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1198 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260926 16:32:32.642856 11299 mini_oidc.cc:152] Starting JWKS server
I20260926 16:32:32.644500 11299 webserver.cc:460] Webserver started at https://127.0.0.1:37403/ using document root <none> and password file <none>
I20260926 16:32:32.644641 11299 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 16:32:32.644876 11299 webserver.cc:460] Webserver started at http://127.0.0.1:43485/ using document root <none> and password file <none>
2026-09-26T16:32:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:32:32Z Disabled control of system clock
I20260926 16:32:32.653685 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:42201
--webserver_interface=127.11.8.254
--webserver_port=0
--builtin_ntp_servers=127.11.8.212:38541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:42201
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37403/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790440324823152-11299-0/testchainca.cert with env {}
W20260926 16:32:32.761719 16433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:32.761982 16433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:32.762020 16433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:32.765895 16433 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:32:32.765944 16433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:32.765959 16433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:32.765976 16433 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:32:32.765991 16433 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 16:32:32.766006 16433 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 16:32:32.766019 16433 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37403/jwks.json
I20260926 16:32:32.771200 16433 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:38541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:42201
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:42201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37403/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790440324823152-11299-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:32.772104 16433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:32.773006 16433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:32.781025 16441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:32.781024 16438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:32.781032 16439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:32.781437 16433 server_base.cc:1034] running on GCE node
I20260926 16:32:32.781991 16433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:32.783114 16433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:32.784271 16433 hybrid_clock.cc:648] HybridClock initialized: now 1790440352784258 us; error 23 us; skew 500 ppm
I20260926 16:32:32.786096 16433 webserver.cc:460] Webserver started at http://127.11.8.254:32999/ using document root <none> and password file <none>
I20260926 16:32:32.786738 16433 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:32.786834 16433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:32.787046 16433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:32.789162 16433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "83f684c2af574207aa2f79729f010f7e"
format_stamp: "Formatted at 2026-09-26 16:32:32 on dist-test-slave-2zvq"
server_key: "c5462087c004defa735164726328e5f4"
server_key_iv: "e876d2d49618e33dcd5fd37502721d31"
server_key_version: "encryptionkey@0"
I20260926 16:32:32.789817 16433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "83f684c2af574207aa2f79729f010f7e"
format_stamp: "Formatted at 2026-09-26 16:32:32 on dist-test-slave-2zvq"
server_key: "c5462087c004defa735164726328e5f4"
server_key_iv: "e876d2d49618e33dcd5fd37502721d31"
server_key_version: "encryptionkey@0"
I20260926 16:32:32.793641 16433 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 16:32:32.796458 16447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:32.797649 16433 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:32:32.797801 16433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "83f684c2af574207aa2f79729f010f7e"
format_stamp: "Formatted at 2026-09-26 16:32:32 on dist-test-slave-2zvq"
server_key: "c5462087c004defa735164726328e5f4"
server_key_iv: "e876d2d49618e33dcd5fd37502721d31"
server_key_version: "encryptionkey@0"
I20260926 16:32:32.797910 16433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:32.813470 16433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:32.814098 16433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:32.814340 16433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:32.821367 16433 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:42201
I20260926 16:32:32.821377 16499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:42201 every 8 connection(s)
I20260926 16:32:32.822371 16433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:32:32.825625 16500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:32.829146 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 16433
I20260926 16:32:32.829262 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:32:32.829543 11299 external_mini_cluster.cc:1428] Setting key ef6c0aadea2ef4d0597b4e584902cfde
I20260926 16:32:32.831727 16500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83f684c2af574207aa2f79729f010f7e: Bootstrap starting.
I20260926 16:32:32.834064 16500 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83f684c2af574207aa2f79729f010f7e: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:32.834988 16500 log.cc:824] T 00000000000000000000000000000000 P 83f684c2af574207aa2f79729f010f7e: Log is configured to *not* fsync() on all Append() calls
I20260926 16:32:32.837301 16500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83f684c2af574207aa2f79729f010f7e: No bootstrap required, opened a new log
I20260926 16:32:32.838943 16503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:32.830923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43960 (local address 127.11.8.254:42201)
0926 16:32:32.831538 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:32.831554 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:32.831589 (+ 35us) server_negotiation.cc:407] Connection header received
0926 16:32:32.832299 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:32.832317 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:32.832717 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:32.832999 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:32.833335 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:32.834415 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:32.835581 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:32.836135 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:32.836251 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:32.837130 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:32.837135 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:32.837148 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:32:32.837582 (+ 434us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:32.837639 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:32:32.837658 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:32.837735 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:32.837896 (+ 161us) server_negotiation.cc:299] Negotiation successful
0926 16:32:32.838183 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":120,"threads_started":1}
I20260926 16:32:32.841034 16500 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83f684c2af574207aa2f79729f010f7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83f684c2af574207aa2f79729f010f7e" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 42201 } }
I20260926 16:32:32.841241 16500 raft_consensus.cc:374] T 00000000000000000000000000000000 P 83f684c2af574207aa2f79729f010f7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:32.841311 16500 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83f684c2af574207aa2f79729f010f7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f684c2af574207aa2f79729f010f7e, State: Initialized, Role: FOLLOWER
I20260926 16:32:32.841657 11299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:32:32.841753 16500 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83f684c2af574207aa2f79729f010f7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83f684c2af574207aa2f79729f010f7e" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 42201 } }
I20260926 16:32:32.841890 16500 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83f684c2af574207aa2f79729f010f7e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:32:32.841946 16500 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83f684c2af574207aa2f79729f010f7e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:32:32.842042 16500 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83f684c2af574207aa2f79729f010f7e [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:32.843035 16500 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83f684c2af574207aa2f79729f010f7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83f684c2af574207aa2f79729f010f7e" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 42201 } }
I20260926 16:32:32.843364 16500 leader_election.cc:302] T 00000000000000000000000000000000 P 83f684c2af574207aa2f79729f010f7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83f684c2af574207aa2f79729f010f7e; no voters:
I20260926 16:32:32.843674 16500 leader_election.cc:288] T 00000000000000000000000000000000 P 83f684c2af574207aa2f79729f010f7e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:32:32.843765 16505 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83f684c2af574207aa2f79729f010f7e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:32:32.844012 16505 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83f684c2af574207aa2f79729f010f7e [term 1 LEADER]: Becoming Leader. State: Replica: 83f684c2af574207aa2f79729f010f7e, State: Running, Role: LEADER
I20260926 16:32:32.844302 16505 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83f684c2af574207aa2f79729f010f7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83f684c2af574207aa2f79729f010f7e" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 42201 } }
I20260926 16:32:32.844820 16500 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83f684c2af574207aa2f79729f010f7e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:32:32.845891 16507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83f684c2af574207aa2f79729f010f7e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83f684c2af574207aa2f79729f010f7e. Latest consensus state: current_term: 1 leader_uuid: "83f684c2af574207aa2f79729f010f7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83f684c2af574207aa2f79729f010f7e" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 42201 } } }
I20260926 16:32:32.846020 16507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83f684c2af574207aa2f79729f010f7e [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:32.846323 16506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83f684c2af574207aa2f79729f010f7e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "83f684c2af574207aa2f79729f010f7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83f684c2af574207aa2f79729f010f7e" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 42201 } } }
I20260926 16:32:32.846433 16506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83f684c2af574207aa2f79729f010f7e [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:32.846380 16514 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:32:32.849466 16514 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:32:32.855068 16514 catalog_manager.cc:1347] Generated new cluster ID: 856c263306de427384ded71f4bf673a1
I20260926 16:32:32.855149 16514 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:32:32.865682 16514 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:32:32.866909 16514 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:32:32.873996 16514 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83f684c2af574207aa2f79729f010f7e: Generated new TSK 0
I20260926 16:32:32.874891 16514 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260926 16:32:32.914249 16530 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260926 16:32:32.914398 16530 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260926 16:32:32.914908 16503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:32.890601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43972 (local address 127.11.8.254:42201)
0926 16:32:32.890742 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:32.890747 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:32.890943 (+ 196us) server_negotiation.cc:407] Connection header received
0926 16:32:32.891169 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:32.891171 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:32.891217 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:32.891296 (+ 79us) server_negotiation.cc:226] Negotiated authn=JWT
0926 16:32:32.891583 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:32.892121 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:32.893370 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:32.893602 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:32.893691 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:32.894070 (+ 379us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 16:32:32.914510 (+ 20440us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37403/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0926 16:32:32.914785 (+ 275us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43972: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37403/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":55}
W20260926 16:32:32.915030 16503 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43972: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37403/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260926 16:32:32.916308 16529 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:32.890514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:42201 (local address 127.0.0.1:43972)
0926 16:32:32.890855 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:32.890870 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:32.891001 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:32.891333 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:32.891336 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:32.891349 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0926 16:32:32.891481 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:32.891488 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:32.892226 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:32.892230 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:32.893209 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:32.893219 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:32.893720 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:32.893724 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:32.893829 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:32.893887 (+ 58us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 16:32:32.914772 (+ 20885us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37403/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0926 16:32:32.915034 (+ 262us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.8.254:42201: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37403/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":120,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.11.8.254:42201: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37403/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260926 16:32:32.917083 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 16433
2026-09-26T16:32:32Z chronyd exiting
I20260926 16:32:32.953636 11299 test_util.cc:175] -----------------------------------------------
I20260926 16:32:32.953723 11299 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790440324823152-11299-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (312 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260926 16:32:32.954818 11299 mini_oidc.cc:152] Starting JWKS server
I20260926 16:32:32.955667 11299 webserver.cc:460] Webserver started at https://127.0.0.1:43141/ using document root <none> and password file <none>
I20260926 16:32:32.955746 11299 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 16:32:32.955950 11299 webserver.cc:460] Webserver started at http://127.0.0.1:34575/ using document root <none> and password file <none>
2026-09-26T16:32:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:32:32Z Disabled control of system clock
I20260926 16:32:32.964874 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:44401
--webserver_interface=127.11.8.254
--webserver_port=0
--builtin_ntp_servers=127.11.8.212:45361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:44401
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43141/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790440324823152-11299-0/testchainca.cert with env {}
W20260926 16:32:33.073745 16536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:33.074023 16536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:33.074066 16536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:33.078066 16536 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:32:33.078121 16536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:33.078137 16536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:33.078153 16536 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:32:33.078168 16536 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 16:32:33.078223 16536 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 16:32:33.078250 16536 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43141/jwks.json
I20260926 16:32:33.083680 16536 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:45361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:44401
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:44401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43141/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790440324823152-11299-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16536
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:33.084683 16536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:33.085646 16536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:33.093988 16544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:33.093976 16541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:33.094000 16542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:33.094178 16536 server_base.cc:1034] running on GCE node
I20260926 16:32:33.094767 16536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:33.095827 16536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:33.097064 16536 hybrid_clock.cc:648] HybridClock initialized: now 1790440353097047 us; error 30 us; skew 500 ppm
I20260926 16:32:33.098976 16536 webserver.cc:460] Webserver started at http://127.11.8.254:36013/ using document root <none> and password file <none>
I20260926 16:32:33.099573 16536 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:33.099635 16536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:33.099828 16536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:33.101763 16536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "444715bef50a478dab2c6d4f87355e24"
format_stamp: "Formatted at 2026-09-26 16:32:33 on dist-test-slave-2zvq"
server_key: "a9309782785adb529138cad9acb425bb"
server_key_iv: "a188f9a3a08a4ef7acb12f10c609d39f"
server_key_version: "encryptionkey@0"
I20260926 16:32:33.102372 16536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "444715bef50a478dab2c6d4f87355e24"
format_stamp: "Formatted at 2026-09-26 16:32:33 on dist-test-slave-2zvq"
server_key: "a9309782785adb529138cad9acb425bb"
server_key_iv: "a188f9a3a08a4ef7acb12f10c609d39f"
server_key_version: "encryptionkey@0"
I20260926 16:32:33.106070 16536 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260926 16:32:33.108654 16550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:33.110164 16536 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 16:32:33.110347 16536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "444715bef50a478dab2c6d4f87355e24"
format_stamp: "Formatted at 2026-09-26 16:32:33 on dist-test-slave-2zvq"
server_key: "a9309782785adb529138cad9acb425bb"
server_key_iv: "a188f9a3a08a4ef7acb12f10c609d39f"
server_key_version: "encryptionkey@0"
I20260926 16:32:33.110476 16536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:33.136565 16536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:33.137259 16536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:33.137462 16536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:33.144388 16536 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:44401
I20260926 16:32:33.144414 16602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:44401 every 8 connection(s)
I20260926 16:32:33.145355 16536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:32:33.148566 16603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:33.150521 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 16536
I20260926 16:32:33.150665 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:32:33.150956 11299 external_mini_cluster.cc:1428] Setting key 831abda85270f178bb12e0f3869e0f91
I20260926 16:32:33.154844 16603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 444715bef50a478dab2c6d4f87355e24: Bootstrap starting.
I20260926 16:32:33.157202 16603 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 444715bef50a478dab2c6d4f87355e24: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:33.157936 16603 log.cc:824] T 00000000000000000000000000000000 P 444715bef50a478dab2c6d4f87355e24: Log is configured to *not* fsync() on all Append() calls
I20260926 16:32:33.159884 16603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 444715bef50a478dab2c6d4f87355e24: No bootstrap required, opened a new log
I20260926 16:32:33.160602 16606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:33.152380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33668 (local address 127.11.8.254:44401)
0926 16:32:33.153086 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:33.153108 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:33.153145 (+ 37us) server_negotiation.cc:407] Connection header received
0926 16:32:33.153979 (+ 834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:33.153999 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:33.154485 (+ 486us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:33.154841 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:33.155273 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:33.156295 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:33.157272 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:33.157782 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:33.157965 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:33.158918 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:33.158924 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:33.158946 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:32:33.159328 (+ 382us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:33.159382 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:32:33.159402 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:33.159518 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:33.159636 (+ 118us) server_negotiation.cc:299] Negotiation successful
0926 16:32:33.159891 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":131,"threads_started":1}
I20260926 16:32:33.163514 11299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:32:33.163919 16603 raft_consensus.cc:348] T 00000000000000000000000000000000 P 444715bef50a478dab2c6d4f87355e24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "444715bef50a478dab2c6d4f87355e24" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 44401 } }
I20260926 16:32:33.164149 16603 raft_consensus.cc:374] T 00000000000000000000000000000000 P 444715bef50a478dab2c6d4f87355e24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:33.164227 16603 raft_consensus.cc:729] T 00000000000000000000000000000000 P 444715bef50a478dab2c6d4f87355e24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444715bef50a478dab2c6d4f87355e24, State: Initialized, Role: FOLLOWER
I20260926 16:32:33.164690 16603 consensus_queue.cc:260] T 00000000000000000000000000000000 P 444715bef50a478dab2c6d4f87355e24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "444715bef50a478dab2c6d4f87355e24" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 44401 } }
I20260926 16:32:33.164858 16603 raft_consensus.cc:388] T 00000000000000000000000000000000 P 444715bef50a478dab2c6d4f87355e24 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:32:33.164925 16603 raft_consensus.cc:482] T 00000000000000000000000000000000 P 444715bef50a478dab2c6d4f87355e24 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:32:33.165021 16603 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 444715bef50a478dab2c6d4f87355e24 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:33.165891 16603 raft_consensus.cc:504] T 00000000000000000000000000000000 P 444715bef50a478dab2c6d4f87355e24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "444715bef50a478dab2c6d4f87355e24" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 44401 } }
I20260926 16:32:33.166162 16603 leader_election.cc:302] T 00000000000000000000000000000000 P 444715bef50a478dab2c6d4f87355e24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 444715bef50a478dab2c6d4f87355e24; no voters:
I20260926 16:32:33.166465 16603 leader_election.cc:288] T 00000000000000000000000000000000 P 444715bef50a478dab2c6d4f87355e24 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:32:33.166608 16608 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 444715bef50a478dab2c6d4f87355e24 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:32:33.166841 16608 raft_consensus.cc:686] T 00000000000000000000000000000000 P 444715bef50a478dab2c6d4f87355e24 [term 1 LEADER]: Becoming Leader. State: Replica: 444715bef50a478dab2c6d4f87355e24, State: Running, Role: LEADER
I20260926 16:32:33.167145 16608 consensus_queue.cc:237] T 00000000000000000000000000000000 P 444715bef50a478dab2c6d4f87355e24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "444715bef50a478dab2c6d4f87355e24" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 44401 } }
I20260926 16:32:33.167613 16603 sys_catalog.cc:564] T 00000000000000000000000000000000 P 444715bef50a478dab2c6d4f87355e24 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:32:33.168692 16610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 444715bef50a478dab2c6d4f87355e24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 444715bef50a478dab2c6d4f87355e24. Latest consensus state: current_term: 1 leader_uuid: "444715bef50a478dab2c6d4f87355e24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "444715bef50a478dab2c6d4f87355e24" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 44401 } } }
I20260926 16:32:33.168820 16610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 444715bef50a478dab2c6d4f87355e24 [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:33.169091 16609 sys_catalog.cc:455] T 00000000000000000000000000000000 P 444715bef50a478dab2c6d4f87355e24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "444715bef50a478dab2c6d4f87355e24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "444715bef50a478dab2c6d4f87355e24" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 44401 } } }
I20260926 16:32:33.169185 16617 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:32:33.169209 16609 sys_catalog.cc:458] T 00000000000000000000000000000000 P 444715bef50a478dab2c6d4f87355e24 [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:33.172362 16617 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:32:33.177667 16617 catalog_manager.cc:1347] Generated new cluster ID: 8bd44732acb746e3ba95ebc10d4861c6
I20260926 16:32:33.177744 16617 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:32:33.192206 16617 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:32:33.193122 16617 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:32:33.204608 16617 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 444715bef50a478dab2c6d4f87355e24: Generated new TSK 0
I20260926 16:32:33.205463 16617 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260926 16:32:33.246191 16633 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260926 16:32:33.246304 16633 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260926 16:32:33.246966 16606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:33.223630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33672 (local address 127.11.8.254:44401)
0926 16:32:33.223764 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:33.223769 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:33.223971 (+ 202us) server_negotiation.cc:407] Connection header received
0926 16:32:33.224141 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:33.224143 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:33.224189 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:33.224269 (+ 80us) server_negotiation.cc:226] Negotiated authn=JWT
0926 16:32:33.224527 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:33.225040 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:33.226260 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:33.226536 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:33.226625 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:33.226966 (+ 341us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 16:32:33.246574 (+ 19608us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43141/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0926 16:32:33.246807 (+ 233us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33672: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43141/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":53}
W20260926 16:32:33.247056 16606 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33672: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43141/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260926 16:32:33.247135 16632 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:33.223576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:44401 (local address 127.0.0.1:33672)
0926 16:32:33.223871 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:33.223887 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:33.224007 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:33.224301 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:33.224304 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:33.224312 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0926 16:32:33.224422 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:33.224429 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:33.225147 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:33.225150 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:33.226142 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:33.226152 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:33.226652 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:33.226656 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:33.226743 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:33.226772 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 16:32:33.246776 (+ 20004us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43141/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0926 16:32:33.246961 (+ 185us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.8.254:44401: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43141/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":114,"threads_started":1}
I20260926 16:32:33.247782 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 16536
2026-09-26T16:32:33Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (331 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260926 16:32:33.286131 11299 mini_oidc.cc:152] Starting JWKS server
I20260926 16:32:33.287070 11299 webserver.cc:460] Webserver started at https://127.0.0.1:33239/ using document root <none> and password file <none>
I20260926 16:32:33.287182 11299 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 16:32:33.287456 11299 webserver.cc:460] Webserver started at http://127.0.0.1:46713/ using document root <none> and password file <none>
2026-09-26T16:32:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:32:33Z Disabled control of system clock
I20260926 16:32:33.296303 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:46169
--webserver_interface=127.11.8.254
--webserver_port=0
--builtin_ntp_servers=127.11.8.212:36499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:46169
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33239/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260926 16:32:33.406203 16639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:33.406522 16639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:33.406564 16639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:33.410459 16639 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:32:33.410509 16639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:33.410524 16639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:33.410542 16639 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:32:33.410557 16639 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 16:32:33.410573 16639 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 16:32:33.410585 16639 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33239/jwks.json
I20260926 16:32:33.415321 16639 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:36499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:46169
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:46169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33239/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:33.416311 16639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:33.417644 16639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:33.425897 16645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:33.426028 16639 server_base.cc:1034] running on GCE node
W20260926 16:32:33.425853 16647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:33.426041 16644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:33.426805 16639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:33.427903 16639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:33.429067 16639 hybrid_clock.cc:648] HybridClock initialized: now 1790440353429054 us; error 29 us; skew 500 ppm
I20260926 16:32:33.431048 16639 webserver.cc:460] Webserver started at http://127.11.8.254:43783/ using document root <none> and password file <none>
I20260926 16:32:33.431644 16639 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:33.431704 16639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:33.431911 16639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:33.433952 16639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "43f0c5ee257447489cf185c1324f2cb0"
format_stamp: "Formatted at 2026-09-26 16:32:33 on dist-test-slave-2zvq"
server_key: "3b1a65d274ee53799b08de54696c1d2f"
server_key_iv: "864a3b1a6c928ac3fba22c64ef262623"
server_key_version: "encryptionkey@0"
I20260926 16:32:33.434543 16639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "43f0c5ee257447489cf185c1324f2cb0"
format_stamp: "Formatted at 2026-09-26 16:32:33 on dist-test-slave-2zvq"
server_key: "3b1a65d274ee53799b08de54696c1d2f"
server_key_iv: "864a3b1a6c928ac3fba22c64ef262623"
server_key_version: "encryptionkey@0"
I20260926 16:32:33.438115 16639 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260926 16:32:33.440528 16653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:33.441749 16639 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:32:33.441884 16639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "43f0c5ee257447489cf185c1324f2cb0"
format_stamp: "Formatted at 2026-09-26 16:32:33 on dist-test-slave-2zvq"
server_key: "3b1a65d274ee53799b08de54696c1d2f"
server_key_iv: "864a3b1a6c928ac3fba22c64ef262623"
server_key_version: "encryptionkey@0"
I20260926 16:32:33.442001 16639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:33.472543 16639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:33.473335 16639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:33.473551 16639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:33.480780 16639 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:46169
I20260926 16:32:33.480789 16705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:46169 every 8 connection(s)
I20260926 16:32:33.481755 16639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:32:33.482193 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 16639
I20260926 16:32:33.482374 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:32:33.482748 11299 external_mini_cluster.cc:1428] Setting key 11304ff85ec47953b122f47e43463705
I20260926 16:32:33.484974 16706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:33.490515 16706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43f0c5ee257447489cf185c1324f2cb0: Bootstrap starting.
I20260926 16:32:33.492094 16709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:33.484117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39012 (local address 127.11.8.254:46169)
0926 16:32:33.484780 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:33.484796 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:33.484832 (+ 36us) server_negotiation.cc:407] Connection header received
0926 16:32:33.485542 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:33.485560 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:33.485967 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:33.486325 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:33.486745 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:33.487824 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:33.488713 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:33.489185 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:33.489365 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:33.490347 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:33.490353 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:33.490366 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:32:33.490792 (+ 426us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:33.490850 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:32:33.490870 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:33.490992 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:33.491144 (+ 152us) server_negotiation.cc:299] Negotiation successful
0926 16:32:33.491385 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":134,"threads_started":1}
I20260926 16:32:33.492748 16706 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43f0c5ee257447489cf185c1324f2cb0: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:33.493494 16706 log.cc:824] T 00000000000000000000000000000000 P 43f0c5ee257447489cf185c1324f2cb0: Log is configured to *not* fsync() on all Append() calls
I20260926 16:32:33.494864 11299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:32:33.495645 16706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43f0c5ee257447489cf185c1324f2cb0: No bootstrap required, opened a new log
I20260926 16:32:33.499043 16706 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43f0c5ee257447489cf185c1324f2cb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43f0c5ee257447489cf185c1324f2cb0" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 46169 } }
I20260926 16:32:33.499253 16706 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43f0c5ee257447489cf185c1324f2cb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:33.499293 16706 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43f0c5ee257447489cf185c1324f2cb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43f0c5ee257447489cf185c1324f2cb0, State: Initialized, Role: FOLLOWER
I20260926 16:32:33.499643 16706 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43f0c5ee257447489cf185c1324f2cb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43f0c5ee257447489cf185c1324f2cb0" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 46169 } }
I20260926 16:32:33.499747 16706 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43f0c5ee257447489cf185c1324f2cb0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:32:33.499783 16706 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43f0c5ee257447489cf185c1324f2cb0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:32:33.499850 16706 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43f0c5ee257447489cf185c1324f2cb0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:33.500654 16706 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43f0c5ee257447489cf185c1324f2cb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43f0c5ee257447489cf185c1324f2cb0" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 46169 } }
I20260926 16:32:33.500928 16706 leader_election.cc:302] T 00000000000000000000000000000000 P 43f0c5ee257447489cf185c1324f2cb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43f0c5ee257447489cf185c1324f2cb0; no voters:
I20260926 16:32:33.501169 16706 leader_election.cc:288] T 00000000000000000000000000000000 P 43f0c5ee257447489cf185c1324f2cb0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:32:33.501374 16711 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43f0c5ee257447489cf185c1324f2cb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:32:33.501643 16711 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43f0c5ee257447489cf185c1324f2cb0 [term 1 LEADER]: Becoming Leader. State: Replica: 43f0c5ee257447489cf185c1324f2cb0, State: Running, Role: LEADER
I20260926 16:32:33.501950 16711 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43f0c5ee257447489cf185c1324f2cb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43f0c5ee257447489cf185c1324f2cb0" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 46169 } }
I20260926 16:32:33.502492 16706 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43f0c5ee257447489cf185c1324f2cb0 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:32:33.503700 16712 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43f0c5ee257447489cf185c1324f2cb0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43f0c5ee257447489cf185c1324f2cb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43f0c5ee257447489cf185c1324f2cb0" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 46169 } } }
I20260926 16:32:33.503823 16712 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43f0c5ee257447489cf185c1324f2cb0 [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:33.504227 16713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43f0c5ee257447489cf185c1324f2cb0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43f0c5ee257447489cf185c1324f2cb0. Latest consensus state: current_term: 1 leader_uuid: "43f0c5ee257447489cf185c1324f2cb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43f0c5ee257447489cf185c1324f2cb0" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 46169 } } }
I20260926 16:32:33.504316 16713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43f0c5ee257447489cf185c1324f2cb0 [sys.catalog]: This master's current role is: LEADER
W20260926 16:32:33.507349 16726 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 43f0c5ee257447489cf185c1324f2cb0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 16:32:33.507443 16726 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 16:32:33.507517 16721 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:32:33.508263 16721 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:32:33.514102 16721 catalog_manager.cc:1347] Generated new cluster ID: 0527adf291344859a5f610d89791533a
I20260926 16:32:33.514192 16721 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:32:33.539762 16721 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:32:33.540665 16721 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:32:33.546805 16721 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43f0c5ee257447489cf185c1324f2cb0: Generated new TSK 0
I20260926 16:32:33.547652 16721 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:32:33.572839 16709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:33.567661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39026 (local address 127.11.8.254:46169)
0926 16:32:33.567810 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:33.567817 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:33.567972 (+ 155us) server_negotiation.cc:407] Connection header received
0926 16:32:33.568103 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:33.568107 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:33.568162 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:33.568280 (+ 118us) server_negotiation.cc:226] Negotiated authn=JWT
0926 16:32:33.568491 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:33.569268 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:33.570161 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:33.570534 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:33.570638 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:33.571031 (+ 393us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 16:32:33.572484 (+ 1453us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33239/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0926 16:32:33.572706 (+ 222us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39026: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33239/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":57}
W20260926 16:32:33.572908 16709 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39026: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33239/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260926 16:32:33.574472 16735 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:33.567540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46169 (local address 127.0.0.1:39026)
0926 16:32:33.567883 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:33.567895 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:33.567995 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:33.568287 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:33.568289 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:33.568296 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0926 16:32:33.568391 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:33.568396 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:33.569397 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:33.569400 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:33.570051 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:33.570058 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:33.570743 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:33.570746 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:33.570814 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:33.570836 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 16:32:33.572645 (+ 1809us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33239/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0926 16:32:33.574303 (+ 1658us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.8.254:46169: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33239/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":229,"mutex_wait_us":1443,"thread_start_us":105,"threads_started":1}
I20260926 16:32:33.574770 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 16639
E20260926 16:32:33.583767 16736 webserver.cc:533] Webserver: sslize failed:
E20260926 16:32:33.583837 16736 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-26T16:32:33Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (327 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-26T16:32:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:32:33Z Disabled control of system clock
I20260926 16:32:33.621901 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:41715
--webserver_interface=127.11.8.254
--webserver_port=0
--builtin_ntp_servers=127.11.8.212:41021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:41715
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790440324823152-11299-0/testchainca.cert with env {}
W20260926 16:32:33.731950 16740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:33.732241 16740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:33.732319 16740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:33.736243 16740 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:32:33.736328 16740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:33.736356 16740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:33.736395 16740 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:32:33.736430 16740 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 16:32:33.736490 16740 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 16:32:33.736521 16740 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260926 16:32:33.741711 16740 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:41021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:41715
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:41715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790440324823152-11299-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16740
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:33.742800 16740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:33.743808 16740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 16:32:33.752419 16740 server_base.cc:1034] running on GCE node
W20260926 16:32:33.752369 16745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:33.752367 16746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:33.752380 16748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:33.753350 16740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:33.754559 16740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:33.755720 16740 hybrid_clock.cc:648] HybridClock initialized: now 1790440353755706 us; error 38 us; skew 500 ppm
I20260926 16:32:33.757676 16740 webserver.cc:460] Webserver started at http://127.11.8.254:37549/ using document root <none> and password file <none>
I20260926 16:32:33.758358 16740 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:33.758427 16740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:33.758647 16740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:33.760736 16740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "558973a7ff964e41901d36debdd9455d"
format_stamp: "Formatted at 2026-09-26 16:32:33 on dist-test-slave-2zvq"
server_key: "7c58e25257c7772dbc648c104d639404"
server_key_iv: "4c08e41e288471115ff527dffb3a1399"
server_key_version: "encryptionkey@0"
I20260926 16:32:33.761344 16740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "558973a7ff964e41901d36debdd9455d"
format_stamp: "Formatted at 2026-09-26 16:32:33 on dist-test-slave-2zvq"
server_key: "7c58e25257c7772dbc648c104d639404"
server_key_iv: "4c08e41e288471115ff527dffb3a1399"
server_key_version: "encryptionkey@0"
I20260926 16:32:33.765056 16740 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260926 16:32:33.767637 16754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:33.768936 16740 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 16:32:33.769098 16740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "558973a7ff964e41901d36debdd9455d"
format_stamp: "Formatted at 2026-09-26 16:32:33 on dist-test-slave-2zvq"
server_key: "7c58e25257c7772dbc648c104d639404"
server_key_iv: "4c08e41e288471115ff527dffb3a1399"
server_key_version: "encryptionkey@0"
I20260926 16:32:33.769214 16740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:33.788650 16740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:33.789433 16740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:33.789637 16740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:33.796484 16740 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:41715
I20260926 16:32:33.796535 16806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:41715 every 8 connection(s)
I20260926 16:32:33.797482 16740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:32:33.797765 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 16740
I20260926 16:32:33.797922 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:32:33.798275 11299 external_mini_cluster.cc:1428] Setting key 5672c8787ded5d07964ea63a6749be2e
I20260926 16:32:33.800781 16807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:33.806322 16807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 558973a7ff964e41901d36debdd9455d: Bootstrap starting.
I20260926 16:32:33.807567 16810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:33.799763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38386 (local address 127.11.8.254:41715)
0926 16:32:33.800376 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:33.800392 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:33.800425 (+ 33us) server_negotiation.cc:407] Connection header received
0926 16:32:33.801124 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:33.801142 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:33.801522 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:33.801848 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:33.802299 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:33.803346 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:33.804243 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:33.804719 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:33.804897 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:33.805805 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:33.805810 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:33.805824 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:32:33.806249 (+ 425us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:33.806362 (+ 113us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:32:33.806400 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:33.806497 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:33.806638 (+ 141us) server_negotiation.cc:299] Negotiation successful
0926 16:32:33.806897 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":134,"threads_started":1}
I20260926 16:32:33.808660 16807 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 558973a7ff964e41901d36debdd9455d: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:33.809547 16807 log.cc:824] T 00000000000000000000000000000000 P 558973a7ff964e41901d36debdd9455d: Log is configured to *not* fsync() on all Append() calls
I20260926 16:32:33.810563 11299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:32:33.811767 16807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 558973a7ff964e41901d36debdd9455d: No bootstrap required, opened a new log
I20260926 16:32:33.815382 16807 raft_consensus.cc:348] T 00000000000000000000000000000000 P 558973a7ff964e41901d36debdd9455d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "558973a7ff964e41901d36debdd9455d" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 41715 } }
I20260926 16:32:33.815609 16807 raft_consensus.cc:374] T 00000000000000000000000000000000 P 558973a7ff964e41901d36debdd9455d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:33.815649 16807 raft_consensus.cc:729] T 00000000000000000000000000000000 P 558973a7ff964e41901d36debdd9455d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558973a7ff964e41901d36debdd9455d, State: Initialized, Role: FOLLOWER
I20260926 16:32:33.815997 16807 consensus_queue.cc:260] T 00000000000000000000000000000000 P 558973a7ff964e41901d36debdd9455d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "558973a7ff964e41901d36debdd9455d" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 41715 } }
I20260926 16:32:33.816104 16807 raft_consensus.cc:388] T 00000000000000000000000000000000 P 558973a7ff964e41901d36debdd9455d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:32:33.816149 16807 raft_consensus.cc:482] T 00000000000000000000000000000000 P 558973a7ff964e41901d36debdd9455d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:32:33.816215 16807 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 558973a7ff964e41901d36debdd9455d [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:33.817085 16807 raft_consensus.cc:504] T 00000000000000000000000000000000 P 558973a7ff964e41901d36debdd9455d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "558973a7ff964e41901d36debdd9455d" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 41715 } }
I20260926 16:32:33.817366 16807 leader_election.cc:302] T 00000000000000000000000000000000 P 558973a7ff964e41901d36debdd9455d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 558973a7ff964e41901d36debdd9455d; no voters:
I20260926 16:32:33.817691 16807 leader_election.cc:288] T 00000000000000000000000000000000 P 558973a7ff964e41901d36debdd9455d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:32:33.817802 16812 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 558973a7ff964e41901d36debdd9455d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:32:33.818089 16812 raft_consensus.cc:686] T 00000000000000000000000000000000 P 558973a7ff964e41901d36debdd9455d [term 1 LEADER]: Becoming Leader. State: Replica: 558973a7ff964e41901d36debdd9455d, State: Running, Role: LEADER
I20260926 16:32:33.818419 16812 consensus_queue.cc:237] T 00000000000000000000000000000000 P 558973a7ff964e41901d36debdd9455d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "558973a7ff964e41901d36debdd9455d" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 41715 } }
I20260926 16:32:33.818938 16807 sys_catalog.cc:564] T 00000000000000000000000000000000 P 558973a7ff964e41901d36debdd9455d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:32:33.820065 16813 sys_catalog.cc:455] T 00000000000000000000000000000000 P 558973a7ff964e41901d36debdd9455d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "558973a7ff964e41901d36debdd9455d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "558973a7ff964e41901d36debdd9455d" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 41715 } } }
I20260926 16:32:33.820191 16813 sys_catalog.cc:458] T 00000000000000000000000000000000 P 558973a7ff964e41901d36debdd9455d [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:33.820480 16814 sys_catalog.cc:455] T 00000000000000000000000000000000 P 558973a7ff964e41901d36debdd9455d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 558973a7ff964e41901d36debdd9455d. Latest consensus state: current_term: 1 leader_uuid: "558973a7ff964e41901d36debdd9455d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "558973a7ff964e41901d36debdd9455d" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 41715 } } }
I20260926 16:32:33.820560 16821 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:32:33.820592 16814 sys_catalog.cc:458] T 00000000000000000000000000000000 P 558973a7ff964e41901d36debdd9455d [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:33.823865 16821 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:32:33.829861 16821 catalog_manager.cc:1347] Generated new cluster ID: d552621ff0ab4a6b964201c54e1a0f2e
I20260926 16:32:33.829943 16821 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:32:33.857035 16821 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:32:33.858187 16821 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:32:33.865582 16821 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 558973a7ff964e41901d36debdd9455d: Generated new TSK 0
I20260926 16:32:33.866459 16821 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:32:33.937073 16810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:33.882571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38398 (local address 127.11.8.254:41715)
0926 16:32:33.882714 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:33.882719 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:33.882867 (+ 148us) server_negotiation.cc:407] Connection header received
0926 16:32:33.883023 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:33.883027 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:33.883083 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:33.883204 (+ 121us) server_negotiation.cc:226] Negotiated authn=JWT
0926 16:32:33.883407 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:33.884203 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:33.885068 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:33.885440 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:33.885547 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:33.885827 (+ 280us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 16:32:33.936522 (+ 50695us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0926 16:32:33.936899 (+ 377us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38398: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":57}
W20260926 16:32:33.937170 16810 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38398: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260926 16:32:33.937546 16836 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:33.882446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:41715 (local address 127.0.0.1:38398)
0926 16:32:33.882771 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:33.882790 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:33.882887 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:33.883204 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:33.883207 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:33.883213 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0926 16:32:33.883305 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:33.883309 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:33.884336 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:33.884338 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:33.884951 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:33.884957 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:33.885560 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:33.885563 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:33.885632 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:33.885653 (+ 21us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 16:32:33.936822 (+ 51169us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0926 16:32:33.937020 (+ 198us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.8.254:41715: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":110,"threads_started":1}
I20260926 16:32:33.938207 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 16740
2026-09-26T16:32:33Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (344 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (123 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (124 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:32:34 dist-test-slave-2zvq krb5kdc[16848](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:32:34 dist-test-slave-2zvq krb5kdc[16848](info): set up 2 sockets
Sep 26 16:32:34 dist-test-slave-2zvq krb5kdc[16848](info): commencing operation
krb5kdc: starting...
W20260926 16:32:35.228266 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:32:35 dist-test-slave-2zvq krb5kdc[16848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:32:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:32:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
I20260926 16:32:35.349073 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:45323
--webserver_interface=127.11.8.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:36261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:45323
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:35.459967 16861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:35.460289 16861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:35.460332 16861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:35.464246 16861 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:32:35.464296 16861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:35.464316 16861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:35.464332 16861 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:32:35.464347 16861 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:32:35.469624 16861 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:36261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:45323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:45323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16861
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:35.470839 16861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:35.471954 16861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:35.480608 16869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:35.480604 16866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:35.480612 16867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:35.480846 16861 server_base.cc:1034] running on GCE node
I20260926 16:32:35.481505 16861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:35.482717 16861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:35.483883 16861 hybrid_clock.cc:648] HybridClock initialized: now 1790440355483850 us; error 45 us; skew 500 ppm
Sep 26 16:32:35 dist-test-slave-2zvq krb5kdc[16848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440355, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:35.486613 16861 init.cc:377] Logged in from keytab as kudu/127.11.8.254@KRBTEST.COM (short username kudu)
I20260926 16:32:35.487490 16861 webserver.cc:460] Webserver started at http://127.11.8.254:40193/ using document root <none> and password file <none>
I20260926 16:32:35.488034 16861 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:35.488119 16861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:35.488346 16861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:35.490470 16861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "838b0bbd8d6449709282390c70874ba0"
format_stamp: "Formatted at 2026-09-26 16:32:35 on dist-test-slave-2zvq"
server_key: "1549044262c9df96ada578ff6ecf4f5c"
server_key_iv: "e69081129e294c19e8621ca7a14c555d"
server_key_version: "encryptionkey@0"
I20260926 16:32:35.491079 16861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "838b0bbd8d6449709282390c70874ba0"
format_stamp: "Formatted at 2026-09-26 16:32:35 on dist-test-slave-2zvq"
server_key: "1549044262c9df96ada578ff6ecf4f5c"
server_key_iv: "e69081129e294c19e8621ca7a14c555d"
server_key_version: "encryptionkey@0"
I20260926 16:32:35.494688 16861 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260926 16:32:35.497066 16876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:35.498478 16861 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:32:35.498603 16861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "838b0bbd8d6449709282390c70874ba0"
format_stamp: "Formatted at 2026-09-26 16:32:35 on dist-test-slave-2zvq"
server_key: "1549044262c9df96ada578ff6ecf4f5c"
server_key_iv: "e69081129e294c19e8621ca7a14c555d"
server_key_version: "encryptionkey@0"
I20260926 16:32:35.498713 16861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:35.515369 16861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:35.518466 16861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:35.518661 16861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:35.525665 16861 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:45323
I20260926 16:32:35.525682 16928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:45323 every 8 connection(s)
I20260926 16:32:35.526669 16861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:32:35.529392 16929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:35.534863 16929 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 838b0bbd8d6449709282390c70874ba0: Bootstrap starting.
I20260926 16:32:35.535652 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 16861
I20260926 16:32:35.535765 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:32:35.536092 11299 external_mini_cluster.cc:1428] Setting key 3f632e6848e3f5bc878f52d544e56576
I20260926 16:32:35.537753 16929 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 838b0bbd8d6449709282390c70874ba0: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:35.538537 16929 log.cc:824] T 00000000000000000000000000000000 P 838b0bbd8d6449709282390c70874ba0: Log is configured to *not* fsync() on all Append() calls
I20260926 16:32:35.540692 16929 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 838b0bbd8d6449709282390c70874ba0: No bootstrap required, opened a new log
Sep 26 16:32:35 dist-test-slave-2zvq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:32:35.545091 16929 raft_consensus.cc:348] T 00000000000000000000000000000000 P 838b0bbd8d6449709282390c70874ba0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "838b0bbd8d6449709282390c70874ba0" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 45323 } }
I20260926 16:32:35.545325 16929 raft_consensus.cc:374] T 00000000000000000000000000000000 P 838b0bbd8d6449709282390c70874ba0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:35.545385 16929 raft_consensus.cc:729] T 00000000000000000000000000000000 P 838b0bbd8d6449709282390c70874ba0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838b0bbd8d6449709282390c70874ba0, State: Initialized, Role: FOLLOWER
I20260926 16:32:35.545825 16929 consensus_queue.cc:260] T 00000000000000000000000000000000 P 838b0bbd8d6449709282390c70874ba0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "838b0bbd8d6449709282390c70874ba0" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 45323 } }
I20260926 16:32:35.545971 16929 raft_consensus.cc:388] T 00000000000000000000000000000000 P 838b0bbd8d6449709282390c70874ba0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:32:35.546056 16929 raft_consensus.cc:482] T 00000000000000000000000000000000 P 838b0bbd8d6449709282390c70874ba0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:32:35.546151 16929 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 838b0bbd8d6449709282390c70874ba0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:35.547257 16929 raft_consensus.cc:504] T 00000000000000000000000000000000 P 838b0bbd8d6449709282390c70874ba0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "838b0bbd8d6449709282390c70874ba0" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 45323 } }
I20260926 16:32:35.547590 16929 leader_election.cc:302] T 00000000000000000000000000000000 P 838b0bbd8d6449709282390c70874ba0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 838b0bbd8d6449709282390c70874ba0; no voters:
I20260926 16:32:35.547895 16929 leader_election.cc:288] T 00000000000000000000000000000000 P 838b0bbd8d6449709282390c70874ba0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:32:35.547978 16934 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 838b0bbd8d6449709282390c70874ba0 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:32:35.548229 16934 raft_consensus.cc:686] T 00000000000000000000000000000000 P 838b0bbd8d6449709282390c70874ba0 [term 1 LEADER]: Becoming Leader. State: Replica: 838b0bbd8d6449709282390c70874ba0, State: Running, Role: LEADER
I20260926 16:32:35.548520 16934 consensus_queue.cc:237] T 00000000000000000000000000000000 P 838b0bbd8d6449709282390c70874ba0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "838b0bbd8d6449709282390c70874ba0" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 45323 } }
I20260926 16:32:35.549273 16929 sys_catalog.cc:564] T 00000000000000000000000000000000 P 838b0bbd8d6449709282390c70874ba0 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:32:35.550032 16936 sys_catalog.cc:455] T 00000000000000000000000000000000 P 838b0bbd8d6449709282390c70874ba0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 838b0bbd8d6449709282390c70874ba0. Latest consensus state: current_term: 1 leader_uuid: "838b0bbd8d6449709282390c70874ba0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "838b0bbd8d6449709282390c70874ba0" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 45323 } } }
I20260926 16:32:35.550150 16936 sys_catalog.cc:458] T 00000000000000000000000000000000 P 838b0bbd8d6449709282390c70874ba0 [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:35.550509 16935 sys_catalog.cc:455] T 00000000000000000000000000000000 P 838b0bbd8d6449709282390c70874ba0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "838b0bbd8d6449709282390c70874ba0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "838b0bbd8d6449709282390c70874ba0" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 45323 } } }
I20260926 16:32:35.550611 16935 sys_catalog.cc:458] T 00000000000000000000000000000000 P 838b0bbd8d6449709282390c70874ba0 [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:35.551510 16932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:35.537248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55948 (local address 127.11.8.254:45323)
0926 16:32:35.537674 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:35.537688 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:35.537721 (+ 33us) server_negotiation.cc:407] Connection header received
0926 16:32:35.538470 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:35.538489 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:35.538883 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:35.539278 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:35.540142 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:35.541155 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:35.542351 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:35.542818 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:35.542939 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:35.545918 (+ 2979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:35.545923 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:35.545935 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:35.545981 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:35.547912 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:35.548469 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:35.548474 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:35.548480 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:35.548611 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:35.548928 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:35.548932 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:35.548934 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:35.549434 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:35.549640 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:35.550025 (+ 385us) server_negotiation.cc:299] Negotiation successful
0926 16:32:35.550400 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":136,"threads_started":1}
I20260926 16:32:35.551545 16943 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:32:35.555336 16943 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:32:35.560635 16943 catalog_manager.cc:1347] Generated new cluster ID: 8d7f5292e95048eb8d0eac1498d0003d
I20260926 16:32:35.560973 16943 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:32:35.575326 16943 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:32:35.576220 16943 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:32:35.585955 16943 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 838b0bbd8d6449709282390c70874ba0: Generated new TSK 0
I20260926 16:32:35.586625 16943 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.8.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab.
Entry for principal kudu/127.11.8.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab.
I20260926 16:32:35.625509 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.193:0
--local_ip_for_outbound_sockets=127.11.8.193
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:45323
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:36261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:35.738735 16955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:35.739006 16955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:35.739049 16955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:35.743009 16955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:35.743062 16955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:35.743119 16955 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.193
I20260926 16:32:35.748296 16955 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:36261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:45323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.193
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:35.749368 16955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:35.750605 16955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:35.760567 16960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:35.760540 16961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:35.760553 16963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:35.760749 16955 server_base.cc:1034] running on GCE node
I20260926 16:32:35.761164 16955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:35.761812 16955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:35.763039 16955 hybrid_clock.cc:648] HybridClock initialized: now 1790440355763026 us; error 35 us; skew 500 ppm
Sep 26 16:32:35 dist-test-slave-2zvq krb5kdc[16848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440355, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:35.765865 16955 init.cc:377] Logged in from keytab as kudu/127.11.8.193@KRBTEST.COM (short username kudu)
I20260926 16:32:35.766783 16955 webserver.cc:460] Webserver started at http://127.11.8.193:45401/ using document root <none> and password file <none>
I20260926 16:32:35.767349 16955 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:35.767437 16955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:35.767668 16955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:35.769614 16955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-0/data/instance:
uuid: "5731499b37ac474e94c2fd81af100dc3"
format_stamp: "Formatted at 2026-09-26 16:32:35 on dist-test-slave-2zvq"
server_key: "ca71120132537dc210f99ce8b48b12b4"
server_key_iv: "683b39e16d265de3c2ec99d53d30b37e"
server_key_version: "encryptionkey@0"
I20260926 16:32:35.770195 16955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance:
uuid: "5731499b37ac474e94c2fd81af100dc3"
format_stamp: "Formatted at 2026-09-26 16:32:35 on dist-test-slave-2zvq"
server_key: "ca71120132537dc210f99ce8b48b12b4"
server_key_iv: "683b39e16d265de3c2ec99d53d30b37e"
server_key_version: "encryptionkey@0"
I20260926 16:32:35.773715 16955 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:32:35.776137 16970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:35.777343 16955 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:32:35.777500 16955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-0/wal
uuid: "5731499b37ac474e94c2fd81af100dc3"
format_stamp: "Formatted at 2026-09-26 16:32:35 on dist-test-slave-2zvq"
server_key: "ca71120132537dc210f99ce8b48b12b4"
server_key_iv: "683b39e16d265de3c2ec99d53d30b37e"
server_key_version: "encryptionkey@0"
I20260926 16:32:35.777612 16955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:35.787147 16955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:35.790374 16955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:35.790588 16955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:35.791209 16955 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:32:35.792071 16955 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:35.792142 16955 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:35.792209 16955 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:35.792258 16955 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:35.801209 16955 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.193:44121
I20260926 16:32:35.801231 17083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.193:44121 every 8 connection(s)
I20260926 16:32:35.802125 16955 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
Sep 26 16:32:35 dist-test-slave-2zvq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440355, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:32:35.811918 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 16955
I20260926 16:32:35.812088 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance
I20260926 16:32:35.812457 11299 external_mini_cluster.cc:1428] Setting key e05b382b187957e83ad3b6c29ea1389e
I20260926 16:32:35.816398 16932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:35.803998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:56433 (local address 127.11.8.254:45323)
0926 16:32:35.804135 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:35.804139 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:35.804995 (+ 856us) server_negotiation.cc:407] Connection header received
0926 16:32:35.806366 (+ 1371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:35.806369 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:35.806424 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:35.806522 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:35.807691 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:35.808222 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:35.809359 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:35.809605 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:35.809719 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:35.812314 (+ 2595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:35.812319 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:35.812321 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:35.812352 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:35.813900 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:35.814529 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:35.814535 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:35.814540 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:35.814637 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:35.815054 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:35.815061 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:35.815065 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:35.815316 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:35.815417 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:35.816050 (+ 633us) server_negotiation.cc:299] Negotiation successful
0926 16:32:35.816192 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 16:32:35.817214 17086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:35.804407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:45323 (local address 127.11.8.193:56433)
0926 16:32:35.804849 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:35.804891 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:35.806117 (+ 1226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:35.806693 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:35.806699 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:35.807068 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:35.807538 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:35.807552 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:35.808349 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:35.808352 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:35.809231 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:35.809240 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:35.809759 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:35.809763 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:35.809938 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:35.810647 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:35.810663 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:35.812105 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:35.814032 (+ 1927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:35.814036 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:35.814047 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:35.814419 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:35.814759 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:35.814762 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:35.814764 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:35.814942 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:35.815466 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:35.815471 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:35.815745 (+ 274us) client_negotiation.cc:769] Sending connection context
0926 16:32:35.815937 (+ 192us) client_negotiation.cc:240] Negotiation successful
0926 16:32:35.816190 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":114,"threads_started":1}
I20260926 16:32:35.818446 17084 heartbeater.cc:347] Connected to a master server at 127.11.8.254:45323
I20260926 16:32:35.818691 17084 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:35.819197 17084 heartbeater.cc:511] Master 127.11.8.254:45323 requested a full tablet report, sending...
I20260926 16:32:35.820705 16893 ts_manager.cc:194] Registered new tserver with Master: 5731499b37ac474e94c2fd81af100dc3 (127.11.8.193:44121)
I20260926 16:32:35.821787 16893 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.193@KRBTEST.COM'} at 127.11.8.193:56433
WARNING: no policy specified for kudu/127.11.8.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab.
Entry for principal kudu/127.11.8.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab.
I20260926 16:32:35.840919 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.194:0
--local_ip_for_outbound_sockets=127.11.8.194
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:45323
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:36261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:35.950985 17089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:35.951303 17089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:35.951385 17089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:35.955484 17089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:35.955580 17089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:35.955679 17089 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.194
I20260926 16:32:35.961241 17089 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:36261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:45323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17089
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.194
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:35.962375 17089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:35.963424 17089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:35.972978 17095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:35.973061 17089 server_base.cc:1034] running on GCE node
W20260926 16:32:35.972955 17097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:35.972955 17094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:35.973590 17089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:35.974144 17089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:35.975327 17089 hybrid_clock.cc:648] HybridClock initialized: now 1790440355975315 us; error 32 us; skew 500 ppm
Sep 26 16:32:35 dist-test-slave-2zvq krb5kdc[16848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440355, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:35.978019 17089 init.cc:377] Logged in from keytab as kudu/127.11.8.194@KRBTEST.COM (short username kudu)
I20260926 16:32:35.978888 17089 webserver.cc:460] Webserver started at http://127.11.8.194:43759/ using document root <none> and password file <none>
I20260926 16:32:35.979429 17089 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:35.979492 17089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:35.979736 17089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:35.981737 17089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-1/data/instance:
uuid: "5344a1f2813046bbb20bc96754f14fed"
format_stamp: "Formatted at 2026-09-26 16:32:35 on dist-test-slave-2zvq"
server_key: "b2ab6695ed9fdd41b0483471632c5d7a"
server_key_iv: "52f1b0074367167912a1017036ddc21e"
server_key_version: "encryptionkey@0"
I20260926 16:32:35.982357 17089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance:
uuid: "5344a1f2813046bbb20bc96754f14fed"
format_stamp: "Formatted at 2026-09-26 16:32:35 on dist-test-slave-2zvq"
server_key: "b2ab6695ed9fdd41b0483471632c5d7a"
server_key_iv: "52f1b0074367167912a1017036ddc21e"
server_key_version: "encryptionkey@0"
I20260926 16:32:35.985868 17089 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260926 16:32:35.988278 17104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:35.989512 17089 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:32:35.989636 17089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-1/wal
uuid: "5344a1f2813046bbb20bc96754f14fed"
format_stamp: "Formatted at 2026-09-26 16:32:35 on dist-test-slave-2zvq"
server_key: "b2ab6695ed9fdd41b0483471632c5d7a"
server_key_iv: "52f1b0074367167912a1017036ddc21e"
server_key_version: "encryptionkey@0"
I20260926 16:32:35.989740 17089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:36.027635 17089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:36.031055 17089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:36.031268 17089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:36.031968 17089 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:32:36.032848 17089 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:36.032896 17089 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:36.032960 17089 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:36.033006 17089 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:36.042084 17089 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.194:35923
I20260926 16:32:36.042111 17217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.194:35923 every 8 connection(s)
I20260926 16:32:36.042990 17089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
I20260926 16:32:36.047212 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 17089
I20260926 16:32:36.047350 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance
I20260926 16:32:36.047624 11299 external_mini_cluster.cc:1428] Setting key 98814cbfc7b5f76b9a621e5b49067750
Sep 26 16:32:36 dist-test-slave-2zvq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440355, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.8.195@KRBTEST.COM; defaulting to no policy
I20260926 16:32:36.058593 16932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:36.044805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:39315 (local address 127.11.8.254:45323)
0926 16:32:36.044969 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:36.044973 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:36.045745 (+ 772us) server_negotiation.cc:407] Connection header received
0926 16:32:36.047121 (+ 1376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:36.047124 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:36.047184 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:36.047288 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:36.048604 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:36.049212 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:36.050943 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:36.051325 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:36.051444 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:36.053932 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:36.053939 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:36.053945 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:36.053979 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:36.056030 (+ 2051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:36.056732 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:36.056736 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:36.056738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:36.056854 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:36.057292 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:36.057296 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:36.057298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:36.057582 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:36.057692 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:36.058352 (+ 660us) server_negotiation.cc:299] Negotiation successful
0926 16:32:36.058465 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 16:32:36.059512 17220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:36.045127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:45323 (local address 127.11.8.194:39315)
0926 16:32:36.045591 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:36.045633 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:36.046908 (+ 1275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:36.047485 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:36.047492 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:36.047891 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:36.048458 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:36.048471 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:36.049330 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:36.049334 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:36.050815 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:36.050830 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:36.051491 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:36.051495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:36.051630 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:36.052264 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:36.052279 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:36.053761 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:36.056145 (+ 2384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:36.056150 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:36.056171 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:36.056551 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:36.056973 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:36.056976 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:36.056978 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:36.057175 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:36.057704 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:36.057711 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:36.058051 (+ 340us) client_negotiation.cc:769] Sending connection context
0926 16:32:36.058302 (+ 251us) client_negotiation.cc:240] Negotiation successful
0926 16:32:36.058613 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":112,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.195@KRBTEST.COM" created.
I20260926 16:32:36.060658 17218 heartbeater.cc:347] Connected to a master server at 127.11.8.254:45323
I20260926 16:32:36.060904 17218 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:36.061484 17218 heartbeater.cc:511] Master 127.11.8.254:45323 requested a full tablet report, sending...
I20260926 16:32:36.062496 16893 ts_manager.cc:194] Registered new tserver with Master: 5344a1f2813046bbb20bc96754f14fed (127.11.8.194:35923)
I20260926 16:32:36.063064 16893 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.194@KRBTEST.COM'} at 127.11.8.194:39315
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab.
Entry for principal kudu/127.11.8.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab.
I20260926 16:32:36.075069 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.195:0
--local_ip_for_outbound_sockets=127.11.8.195
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:45323
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:36261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:36.186540 17223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:36.186774 17223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:36.186841 17223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:36.190899 17223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:36.190963 17223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:36.191056 17223 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.195
I20260926 16:32:36.196388 17223 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:36261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:45323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17223
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.195
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:36.197459 17223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:36.198513 17223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:36.208226 17228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:36.208218 17231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:36.208272 17223 server_base.cc:1034] running on GCE node
W20260926 16:32:36.208218 17229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:36.208876 17223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:36.209471 17223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:36.210665 17223 hybrid_clock.cc:648] HybridClock initialized: now 1790440356210650 us; error 25 us; skew 500 ppm
Sep 26 16:32:36 dist-test-slave-2zvq krb5kdc[16848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440356, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:36.213429 17223 init.cc:377] Logged in from keytab as kudu/127.11.8.195@KRBTEST.COM (short username kudu)
I20260926 16:32:36.214241 17223 webserver.cc:460] Webserver started at http://127.11.8.195:35767/ using document root <none> and password file <none>
I20260926 16:32:36.214838 17223 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:36.214927 17223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:36.215149 17223 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:36.217144 17223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-2/data/instance:
uuid: "a67187f8f7e24fa8a78b2662428ec515"
format_stamp: "Formatted at 2026-09-26 16:32:36 on dist-test-slave-2zvq"
server_key: "9a2cac39fd888e07f743662b76cc2366"
server_key_iv: "a03a3b62db355b54924e0f5fd10ec370"
server_key_version: "encryptionkey@0"
I20260926 16:32:36.217757 17223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance:
uuid: "a67187f8f7e24fa8a78b2662428ec515"
format_stamp: "Formatted at 2026-09-26 16:32:36 on dist-test-slave-2zvq"
server_key: "9a2cac39fd888e07f743662b76cc2366"
server_key_iv: "a03a3b62db355b54924e0f5fd10ec370"
server_key_version: "encryptionkey@0"
I20260926 16:32:36.221767 17223 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260926 16:32:36.224207 17238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:36.225390 17223 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260926 16:32:36.225520 17223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-2/wal
uuid: "a67187f8f7e24fa8a78b2662428ec515"
format_stamp: "Formatted at 2026-09-26 16:32:36 on dist-test-slave-2zvq"
server_key: "9a2cac39fd888e07f743662b76cc2366"
server_key_iv: "a03a3b62db355b54924e0f5fd10ec370"
server_key_version: "encryptionkey@0"
I20260926 16:32:36.225631 17223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:36.255934 17223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:36.259246 17223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:36.259452 17223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:36.260042 17223 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:32:36.260914 17223 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:36.260982 17223 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:36.261054 17223 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:36.261098 17223 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:36.269981 17223 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.195:38265
I20260926 16:32:36.270001 17351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.195:38265 every 8 connection(s)
I20260926 16:32:36.271307 17223 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
I20260926 16:32:36.281184 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 17223
I20260926 16:32:36.281337 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance
Sep 26 16:32:36 dist-test-slave-2zvq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440356, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:32:36.281652 11299 external_mini_cluster.cc:1428] Setting key b0068613d7a2a42ddd694c015ce6094c
I20260926 16:32:36.285750 16932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:36.273179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:41217 (local address 127.11.8.254:45323)
0926 16:32:36.273323 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:36.273334 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:36.274200 (+ 866us) server_negotiation.cc:407] Connection header received
0926 16:32:36.275674 (+ 1474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:36.275678 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:36.275734 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:36.275833 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:36.277128 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:36.277974 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:36.279226 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:36.279463 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:36.279573 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:36.282408 (+ 2835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:36.282411 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:36.282414 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:36.282442 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:36.284057 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:36.284507 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:36.284509 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:36.284511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:36.284581 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:36.284855 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:36.284857 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:36.284858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:36.285045 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:36.285121 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:36.285553 (+ 432us) server_negotiation.cc:299] Negotiation successful
0926 16:32:36.285639 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 16:32:36.286675 17354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:36.273561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:45323 (local address 127.11.8.195:41217)
0926 16:32:36.274050 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:36.274093 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:36.275428 (+ 1335us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:36.276013 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:36.276020 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:36.276366 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:36.276964 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:36.276987 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:36.278096 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:36.278100 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:36.279090 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:36.279100 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:36.279612 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:36.279615 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:36.279795 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:36.280405 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:36.280421 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:36.282143 (+ 1722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:36.284153 (+ 2010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:36.284157 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:36.284168 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:36.284421 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:36.284656 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:36.284657 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:36.284659 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:36.284780 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:36.285137 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:36.285142 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:36.285354 (+ 212us) client_negotiation.cc:769] Sending connection context
0926 16:32:36.285509 (+ 155us) client_negotiation.cc:240] Negotiation successful
0926 16:32:36.285739 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":114,"threads_started":1}
I20260926 16:32:36.287806 17352 heartbeater.cc:347] Connected to a master server at 127.11.8.254:45323
I20260926 16:32:36.288072 17352 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:36.288582 17352 heartbeater.cc:511] Master 127.11.8.254:45323 requested a full tablet report, sending...
I20260926 16:32:36.289505 16893 ts_manager.cc:194] Registered new tserver with Master: a67187f8f7e24fa8a78b2662428ec515 (127.11.8.195:38265)
I20260926 16:32:36.290074 16893 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.195@KRBTEST.COM'} at 127.11.8.195:41217
I20260926 16:32:36.295657 11299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/krb5kdc/test-admin.keytab.
Sep 26 16:32:36 dist-test-slave-2zvq krb5kdc[16848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:36.311216 11299 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260926 16:32:36.313979 17359 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:36.312610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60454 (local address 127.11.8.193:44121)
0926 16:32:36.312924 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:36.312938 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:36.312963 (+ 25us) server_negotiation.cc:407] Connection header received
0926 16:32:36.313037 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:36.313042 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:36.313228 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:36.313347 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:36.313810 (+ 463us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60454: BlockingRecv error: recv got EOF from 127.0.0.1:60454 (error 108)
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":124,"threads_started":1}
W20260926 16:32:36.314419 11299 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/krb5kdc/krb5cc' of type FILE
Sep 26 16:32:36 dist-test-slave-2zvq krb5kdc[16848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:36.315413 11299 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 16:32:36 dist-test-slave-2zvq krb5kdc[16848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:36.316365 11299 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 16:32:36 dist-test-slave-2zvq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.193@KRBTEST.COM
I20260926 16:32:36.325330 17359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:36.317218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60456 (local address 127.11.8.193:44121)
0926 16:32:36.317316 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:36.317319 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:36.317448 (+ 129us) server_negotiation.cc:407] Connection header received
0926 16:32:36.317525 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:36.317527 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:36.317560 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:36.317623 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:36.318130 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:36.318976 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:36.319772 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:36.320154 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:36.320234 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:36.322416 (+ 2182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:36.322425 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:36.322438 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:36.322460 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:36.323914 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:36.324302 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:36.324307 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:36.324308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:36.324379 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:36.324601 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:36.324604 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:36.324605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:36.324813 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:36.324952 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:36.325105 (+ 153us) server_negotiation.cc:299] Negotiation successful
0926 16:32:36.325218 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
W20260926 16:32:36.329156 17359 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:36.328227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60458 (local address 127.11.8.193:44121)
0926 16:32:36.328359 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:36.328362 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:36.328397 (+ 35us) server_negotiation.cc:407] Connection header received
0926 16:32:36.328510 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:36.328512 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:36.328546 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:36.328607 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:36.329010 (+ 403us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60458: BlockingRecv error: recv got EOF from 127.0.0.1:60458 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260926 16:32:36.329241 11299 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/krb5kdc/krb5cc' of type FILE
Sep 26 16:32:36 dist-test-slave-2zvq krb5kdc[16848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:36.330255 11299 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 16:32:36 dist-test-slave-2zvq krb5kdc[16848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:36.331205 11299 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 16:32:36 dist-test-slave-2zvq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.193@KRBTEST.COM
I20260926 16:32:36.339159 17359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:36.331964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60466 (local address 127.11.8.193:44121)
0926 16:32:36.332103 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:36.332107 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:36.332193 (+ 86us) server_negotiation.cc:407] Connection header received
0926 16:32:36.332339 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:36.332345 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:36.332388 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:36.332491 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:36.332966 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:36.333453 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:36.334240 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:36.334516 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:36.334623 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:36.336647 (+ 2024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:36.336650 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:36.336652 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:36.336672 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:36.337831 (+ 1159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:36.338194 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:36.338196 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:36.338198 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:36.338317 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:36.338557 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:36.338559 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:36.338560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:36.338697 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:36.338803 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:36.338979 (+ 176us) server_negotiation.cc:299] Negotiation successful
0926 16:32:36.339060 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260926 16:32:36.341858 17359 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:36.340673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60478 (local address 127.11.8.193:44121)
0926 16:32:36.340792 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:36.340797 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:36.340830 (+ 33us) server_negotiation.cc:407] Connection header received
0926 16:32:36.341006 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:36.341010 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:36.341091 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:36.341177 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:36.341706 (+ 529us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60478: BlockingRecv error: recv got EOF from 127.0.0.1:60478 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260926 16:32:36.341846 11299 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790440324823152-11299-0/krb5kdc/krb5cc' of type FILE
Sep 26 16:32:36 dist-test-slave-2zvq krb5kdc[16848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:36.342942 11299 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 16:32:36 dist-test-slave-2zvq krb5kdc[16848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:36.343813 11299 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 16:32:36 dist-test-slave-2zvq krb5kdc[16848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.193@KRBTEST.COM
I20260926 16:32:36.352408 17359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:36.344509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60488 (local address 127.11.8.193:44121)
0926 16:32:36.344611 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:36.344615 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:36.344756 (+ 141us) server_negotiation.cc:407] Connection header received
0926 16:32:36.344875 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:36.344878 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:36.344918 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:36.345033 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:36.345494 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:36.345962 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:36.346834 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:36.347052 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:36.347161 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:36.349357 (+ 2196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:36.349361 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:36.349364 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:36.349392 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:36.350926 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:36.351362 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:36.351364 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:36.351366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:36.351435 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:36.351663 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:36.351665 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:36.351667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:36.351819 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:36.351924 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:36.352113 (+ 189us) server_negotiation.cc:299] Negotiation successful
0926 16:32:36.352226 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 16:32:36.353518 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 16955
I20260926 16:32:36.359083 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 17089
I20260926 16:32:36.364353 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 17223
I20260926 16:32:36.369786 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 16861
2026-09-26T16:32:36Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2186 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:32:36 dist-test-slave-2zvq krb5kdc[17372](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:32:36 dist-test-slave-2zvq krb5kdc[17372](info): set up 2 sockets
Sep 26 16:32:36 dist-test-slave-2zvq krb5kdc[17372](info): commencing operation
krb5kdc: starting...
W20260926 16:32:37.408488 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:32:37 dist-test-slave-2zvq krb5kdc[17372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:32:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:32:37Z Disabled control of system clock
WARNING: no policy specified for oryx/127.11.8.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.8.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.8.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.254.keytab.
Entry for principal oryx/127.11.8.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.254.keytab.
I20260926 16:32:37.527275 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:32843
--webserver_interface=127.11.8.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.254.keytab
--principal=oryx/127.11.8.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:40813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:32843
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:37.636427 17385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:37.636709 17385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:37.636751 17385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:37.640725 17385 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:32:37.640777 17385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:37.640797 17385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:37.640815 17385 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:32:37.640829 17385 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:32:37.646086 17385 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:40813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:32843
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.254.keytab
--principal=oryx/127.11.8.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:32843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:37.647415 17385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:37.648366 17385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:37.656684 17390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:37.656836 17385 server_base.cc:1034] running on GCE node
W20260926 16:32:37.656708 17393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:37.657009 17391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:37.657581 17385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:37.658681 17385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:37.659842 17385 hybrid_clock.cc:648] HybridClock initialized: now 1790440357659780 us; error 77 us; skew 500 ppm
Sep 26 16:32:37 dist-test-slave-2zvq krb5kdc[17372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440357, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:37.662595 17385 init.cc:377] Logged in from keytab as oryx/127.11.8.254@KRBTEST.COM (short username oryx)
I20260926 16:32:37.663371 17385 webserver.cc:460] Webserver started at http://127.11.8.254:33873/ using document root <none> and password file <none>
I20260926 16:32:37.663874 17385 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:37.663929 17385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:37.664095 17385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:37.666127 17385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "d1aa72ae6508448486ff84262f3af952"
format_stamp: "Formatted at 2026-09-26 16:32:37 on dist-test-slave-2zvq"
server_key: "ddc85e8c3ce5468959c5e8fe0f389aa6"
server_key_iv: "25cfc3a0702b06298273cf0f349192aa"
server_key_version: "encryptionkey@0"
I20260926 16:32:37.666771 17385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "d1aa72ae6508448486ff84262f3af952"
format_stamp: "Formatted at 2026-09-26 16:32:37 on dist-test-slave-2zvq"
server_key: "ddc85e8c3ce5468959c5e8fe0f389aa6"
server_key_iv: "25cfc3a0702b06298273cf0f349192aa"
server_key_version: "encryptionkey@0"
I20260926 16:32:37.670572 17385 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:32:37.673086 17400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:37.674407 17385 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:32:37.674538 17385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "d1aa72ae6508448486ff84262f3af952"
format_stamp: "Formatted at 2026-09-26 16:32:37 on dist-test-slave-2zvq"
server_key: "ddc85e8c3ce5468959c5e8fe0f389aa6"
server_key_iv: "25cfc3a0702b06298273cf0f349192aa"
server_key_version: "encryptionkey@0"
I20260926 16:32:37.674654 17385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:37.691871 17385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:37.694834 17385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:37.695041 17385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:37.701828 17385 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:32843
I20260926 16:32:37.702112 17452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:32843 every 8 connection(s)
I20260926 16:32:37.702960 17385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:32:37.703121 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 17385
I20260926 16:32:37.703251 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:32:37.703646 11299 external_mini_cluster.cc:1428] Setting key f7e274a616cf6ca373efc2d42512b08c
I20260926 16:32:37.706372 17453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 26 16:32:37 dist-test-slave-2zvq krb5kdc[17372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM
I20260926 16:32:37.714020 17453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952: Bootstrap starting.
I20260926 16:32:37.716431 17453 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:37.717249 17453 log.cc:824] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952: Log is configured to *not* fsync() on all Append() calls
I20260926 16:32:37.718715 17456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:37.705116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38456 (local address 127.11.8.254:32843)
0926 16:32:37.705511 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:37.705529 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:37.705576 (+ 47us) server_negotiation.cc:407] Connection header received
0926 16:32:37.706196 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:37.706213 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:37.706652 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:37.706933 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:37.707914 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:37.708901 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:37.710167 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:37.710724 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:37.710846 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:37.713817 (+ 2971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:37.713822 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:37.713836 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:37.713865 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:37.715662 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:37.716124 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:37.716134 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:37.716139 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:37.716250 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:37.716537 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:37.716543 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:37.716546 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:37.716935 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:37.717142 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:37.717421 (+ 279us) server_negotiation.cc:299] Negotiation successful
0926 16:32:37.717741 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":133,"threads_started":1}
I20260926 16:32:37.719712 17453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952: No bootstrap required, opened a new log
I20260926 16:32:37.723592 17453 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } }
I20260926 16:32:37.723857 17453 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:37.723984 17453 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1aa72ae6508448486ff84262f3af952, State: Initialized, Role: FOLLOWER
I20260926 16:32:37.724514 17453 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } }
I20260926 16:32:37.724677 17453 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:32:37.724774 17453 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:32:37.724920 17453 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:37.725850 17453 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } }
I20260926 16:32:37.726209 17453 leader_election.cc:302] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1aa72ae6508448486ff84262f3af952; no voters:
I20260926 16:32:37.726567 17453 leader_election.cc:288] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:32:37.726670 17458 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:32:37.726882 17458 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [term 1 LEADER]: Becoming Leader. State: Replica: d1aa72ae6508448486ff84262f3af952, State: Running, Role: LEADER
I20260926 16:32:37.727182 17458 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } }
I20260926 16:32:37.727860 17453 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:32:37.728817 17460 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1aa72ae6508448486ff84262f3af952. Latest consensus state: current_term: 1 leader_uuid: "d1aa72ae6508448486ff84262f3af952" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } }
I20260926 16:32:37.728976 17460 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:37.728817 17459 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d1aa72ae6508448486ff84262f3af952" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } }
I20260926 16:32:37.729076 17459 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:37.729344 17467 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:32:37.732095 17467 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:32:37.737843 17467 catalog_manager.cc:1347] Generated new cluster ID: 2d3a2295b7af482ea4f34e28e17b0044
I20260926 16:32:37.737912 17467 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:32:37.765290 17467 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:32:37.766294 17467 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:32:37.775208 17467 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952: Generated new TSK 0
I20260926 16:32:37.775903 17467 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.11.8.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.8.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.8.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.193.keytab.
Entry for principal oryx/127.11.8.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.193.keytab.
I20260926 16:32:37.806789 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.193:0
--local_ip_for_outbound_sockets=127.11.8.193
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:32843
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.193.keytab
--principal=oryx/127.11.8.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:40813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:37.918458 17479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:37.918723 17479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:37.918766 17479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:37.922933 17479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:37.922986 17479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:37.923071 17479 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.193
I20260926 16:32:37.928319 17479 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:40813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.193.keytab
--principal=oryx/127.11.8.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:32843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.193
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:37.929359 17479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:37.930395 17479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:37.940079 17484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:37.940079 17485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:37.940236 17479 server_base.cc:1034] running on GCE node
W20260926 16:32:37.940055 17487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:37.940677 17479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:37.941298 17479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:37.942456 17479 hybrid_clock.cc:648] HybridClock initialized: now 1790440357942441 us; error 28 us; skew 500 ppm
Sep 26 16:32:37 dist-test-slave-2zvq krb5kdc[17372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440357, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:37.945139 17479 init.cc:377] Logged in from keytab as oryx/127.11.8.193@KRBTEST.COM (short username oryx)
I20260926 16:32:37.945950 17479 webserver.cc:460] Webserver started at http://127.11.8.193:37679/ using document root <none> and password file <none>
I20260926 16:32:37.946506 17479 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:37.946568 17479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:37.946769 17479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:37.948774 17479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-0/data/instance:
uuid: "452825c51b3246d1acaef2b0bed22348"
format_stamp: "Formatted at 2026-09-26 16:32:37 on dist-test-slave-2zvq"
server_key: "922bb905b6b8fbbcb9dc95413ee1cfba"
server_key_iv: "b15f2d6999cb733941094d9a23956d64"
server_key_version: "encryptionkey@0"
I20260926 16:32:37.949345 17479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance:
uuid: "452825c51b3246d1acaef2b0bed22348"
format_stamp: "Formatted at 2026-09-26 16:32:37 on dist-test-slave-2zvq"
server_key: "922bb905b6b8fbbcb9dc95413ee1cfba"
server_key_iv: "b15f2d6999cb733941094d9a23956d64"
server_key_version: "encryptionkey@0"
I20260926 16:32:37.953161 17479 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260926 16:32:37.955710 17494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:37.956990 17479 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 16:32:37.957130 17479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-0/wal
uuid: "452825c51b3246d1acaef2b0bed22348"
format_stamp: "Formatted at 2026-09-26 16:32:37 on dist-test-slave-2zvq"
server_key: "922bb905b6b8fbbcb9dc95413ee1cfba"
server_key_iv: "b15f2d6999cb733941094d9a23956d64"
server_key_version: "encryptionkey@0"
I20260926 16:32:37.957245 17479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:37.989415 17479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:37.992936 17479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:37.993158 17479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:37.993786 17479 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:32:37.994742 17479 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:37.994818 17479 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:37.994892 17479 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:37.994935 17479 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:38.003789 17479 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.193:42459
I20260926 16:32:38.003818 17607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.193:42459 every 8 connection(s)
I20260926 16:32:38.004689 17479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
I20260926 16:32:38.013051 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 17479
I20260926 16:32:38.013204 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance
I20260926 16:32:38.013598 11299 external_mini_cluster.cc:1428] Setting key b801932f9c92d19693f6bf6b14cbe590
Sep 26 16:32:38 dist-test-slave-2zvq krb5kdc[17372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440357, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.193@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM
I20260926 16:32:38.018406 17456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:38.006606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:60103 (local address 127.11.8.254:32843)
0926 16:32:38.006743 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:38.006746 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:38.007577 (+ 831us) server_negotiation.cc:407] Connection header received
0926 16:32:38.008859 (+ 1282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:38.008862 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:38.008910 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:38.009012 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:38.010102 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:38.010691 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:38.011754 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:38.011970 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:38.012053 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:38.014638 (+ 2585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:38.014644 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:38.014651 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:38.014677 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:38.016353 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:38.016880 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:38.016884 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:38.016886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:38.016967 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:38.017297 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:38.017302 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:38.017304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:38.017490 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:38.017598 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:38.018087 (+ 489us) server_negotiation.cc:299] Negotiation successful
0926 16:32:38.018214 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 16:32:38.019253 17610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:38.006986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.11.8.193:60103)
0926 16:32:38.007414 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:38.007467 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:38.008661 (+ 1194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:38.009163 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:38.009169 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:38.009523 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:38.009980 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:38.009993 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:38.010796 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:38.010799 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:38.011654 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:38.011662 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:38.012100 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:38.012102 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:38.012235 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:38.012731 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:38.012747 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:38.014481 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:38.016473 (+ 1992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:38.016477 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:38.016487 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:38.016776 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:38.017059 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:38.017064 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:38.017066 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:38.017204 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:38.017594 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:38.017600 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:38.017835 (+ 235us) client_negotiation.cc:769] Sending connection context
0926 16:32:38.018048 (+ 213us) client_negotiation.cc:240] Negotiation successful
0926 16:32:38.018373 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":106,"threads_started":1}
I20260926 16:32:38.020401 17608 heartbeater.cc:347] Connected to a master server at 127.11.8.254:32843
I20260926 16:32:38.020641 17608 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:38.021093 17608 heartbeater.cc:511] Master 127.11.8.254:32843 requested a full tablet report, sending...
I20260926 16:32:38.022753 17417 ts_manager.cc:194] Registered new tserver with Master: 452825c51b3246d1acaef2b0bed22348 (127.11.8.193:42459)
WARNING: no policy specified for oryx/127.11.8.194@KRBTEST.COM; defaulting to no policy
I20260926 16:32:38.024066 17417 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.8.193@KRBTEST.COM'} at 127.11.8.193:60103
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.8.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.8.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.194.keytab.
Entry for principal oryx/127.11.8.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.194.keytab.
I20260926 16:32:38.042191 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.194:0
--local_ip_for_outbound_sockets=127.11.8.194
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:32843
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.194.keytab
--principal=oryx/127.11.8.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:40813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:38.159025 17613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:38.159291 17613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:38.159336 17613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:38.163488 17613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:38.163546 17613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:38.163611 17613 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.194
I20260926 16:32:38.168794 17613 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:40813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.194.keytab
--principal=oryx/127.11.8.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:32843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17613
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.194
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:38.169715 17613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:38.170761 17613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 16:32:38.180109 17613 server_base.cc:1034] running on GCE node
W20260926 16:32:38.180074 17618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:38.180074 17621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:38.180064 17619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:38.180760 17613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:38.181329 17613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:38.182519 17613 hybrid_clock.cc:648] HybridClock initialized: now 1790440358182488 us; error 43 us; skew 500 ppm
Sep 26 16:32:38 dist-test-slave-2zvq krb5kdc[17372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440358, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:38.185456 17613 init.cc:377] Logged in from keytab as oryx/127.11.8.194@KRBTEST.COM (short username oryx)
I20260926 16:32:38.186618 17613 webserver.cc:460] Webserver started at http://127.11.8.194:45933/ using document root <none> and password file <none>
I20260926 16:32:38.187157 17613 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:38.187249 17613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:38.187489 17613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:38.189436 17613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-1/data/instance:
uuid: "beada9654a9f49fa98d63053a48a77da"
format_stamp: "Formatted at 2026-09-26 16:32:38 on dist-test-slave-2zvq"
server_key: "8e5fd1cc4338e86425ce5c0007aee19d"
server_key_iv: "1bdc62d301b2bd35e5101c2cca019f6c"
server_key_version: "encryptionkey@0"
I20260926 16:32:38.189991 17613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance:
uuid: "beada9654a9f49fa98d63053a48a77da"
format_stamp: "Formatted at 2026-09-26 16:32:38 on dist-test-slave-2zvq"
server_key: "8e5fd1cc4338e86425ce5c0007aee19d"
server_key_iv: "1bdc62d301b2bd35e5101c2cca019f6c"
server_key_version: "encryptionkey@0"
I20260926 16:32:38.193753 17613 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260926 16:32:38.196478 17628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:38.197901 17613 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260926 16:32:38.198045 17613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-1/wal
uuid: "beada9654a9f49fa98d63053a48a77da"
format_stamp: "Formatted at 2026-09-26 16:32:38 on dist-test-slave-2zvq"
server_key: "8e5fd1cc4338e86425ce5c0007aee19d"
server_key_iv: "1bdc62d301b2bd35e5101c2cca019f6c"
server_key_version: "encryptionkey@0"
I20260926 16:32:38.198168 17613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:38.216357 17613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:38.219944 17613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:38.220157 17613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:38.220772 17613 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:32:38.221642 17613 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:38.221726 17613 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:38.221802 17613 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:38.221861 17613 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:38.231564 17613 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.194:45431
I20260926 16:32:38.231582 17741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.194:45431 every 8 connection(s)
I20260926 16:32:38.232630 17613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
I20260926 16:32:38.238536 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 17613
I20260926 16:32:38.238651 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance
I20260926 16:32:38.238969 11299 external_mini_cluster.cc:1428] Setting key a475fbe66912c24e0fe4762a2d84cbb7
Sep 26 16:32:38 dist-test-slave-2zvq krb5kdc[17372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440358, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.194@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM
I20260926 16:32:38.248236 17456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:38.234688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:51409 (local address 127.11.8.254:32843)
0926 16:32:38.234835 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:38.234839 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:38.235716 (+ 877us) server_negotiation.cc:407] Connection header received
0926 16:32:38.237140 (+ 1424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:38.237143 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:38.237190 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:38.237267 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:38.238648 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:38.239398 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:38.240695 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:38.241077 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:38.241203 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:38.244005 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:38.244011 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:38.244017 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:38.244050 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:38.245934 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:38.246523 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:38.246527 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:38.246529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:38.246631 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:38.247021 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:38.247024 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:38.247026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:38.247265 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:38.247385 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:38.247890 (+ 505us) server_negotiation.cc:299] Negotiation successful
0926 16:32:38.248027 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
WARNING: no policy specified for oryx/127.11.8.195@KRBTEST.COM; defaulting to no policy
I20260926 16:32:38.249025 17744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:38.235050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.11.8.194:51409)
0926 16:32:38.235558 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:38.235600 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:38.236945 (+ 1345us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:38.237478 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:38.237486 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:38.237881 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:38.238492 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:38.238510 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:38.239563 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:38.239567 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:38.240576 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:38.240585 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:38.241275 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:38.241278 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:38.241621 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:38.242360 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:38.242376 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:38.243839 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:38.246067 (+ 2228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:38.246071 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:38.246082 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:38.246417 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:38.246761 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:38.246763 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:38.246765 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:38.246903 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:38.247381 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:38.247386 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:38.247623 (+ 237us) client_negotiation.cc:769] Sending connection context
0926 16:32:38.247777 (+ 154us) client_negotiation.cc:240] Negotiation successful
0926 16:32:38.248032 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":113,"threads_started":1}
I20260926 16:32:38.250332 17742 heartbeater.cc:347] Connected to a master server at 127.11.8.254:32843
I20260926 16:32:38.250592 17742 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:38.251149 17742 heartbeater.cc:511] Master 127.11.8.254:32843 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.8.195@KRBTEST.COM" created.
I20260926 16:32:38.252238 17417 ts_manager.cc:194] Registered new tserver with Master: beada9654a9f49fa98d63053a48a77da (127.11.8.194:45431)
I20260926 16:32:38.252805 17417 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.8.194@KRBTEST.COM'} at 127.11.8.194:51409
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.8.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.195.keytab.
Entry for principal oryx/127.11.8.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.195.keytab.
I20260926 16:32:38.265398 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.195:0
--local_ip_for_outbound_sockets=127.11.8.195
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:32843
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.195.keytab
--principal=oryx/127.11.8.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:40813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:38.373407 17747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:38.373687 17747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:38.373762 17747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:38.377966 17747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:38.378033 17747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:38.378139 17747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.195
I20260926 16:32:38.383322 17747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:40813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.195.keytab
--principal=oryx/127.11.8.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:32843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.195
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:38.384302 17747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:38.385337 17747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:38.394982 17753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:38.394975 17752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:38.395051 17747 server_base.cc:1034] running on GCE node
W20260926 16:32:38.394981 17755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:38.395706 17747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:38.396407 17747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:38.397590 17747 hybrid_clock.cc:648] HybridClock initialized: now 1790440358397551 us; error 50 us; skew 500 ppm
Sep 26 16:32:38 dist-test-slave-2zvq krb5kdc[17372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440358, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:38.400506 17747 init.cc:377] Logged in from keytab as oryx/127.11.8.195@KRBTEST.COM (short username oryx)
I20260926 16:32:38.401415 17747 webserver.cc:460] Webserver started at http://127.11.8.195:34649/ using document root <none> and password file <none>
I20260926 16:32:38.402051 17747 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:38.402113 17747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:38.402410 17747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:38.404446 17747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-2/data/instance:
uuid: "fe17e60ac69141458c23a427d88a8ca9"
format_stamp: "Formatted at 2026-09-26 16:32:38 on dist-test-slave-2zvq"
server_key: "2adf451f83c6e0c0f8169a05b2cb6554"
server_key_iv: "5f7a2d901d9f94fa28a9acdf406af69c"
server_key_version: "encryptionkey@0"
I20260926 16:32:38.405025 17747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance:
uuid: "fe17e60ac69141458c23a427d88a8ca9"
format_stamp: "Formatted at 2026-09-26 16:32:38 on dist-test-slave-2zvq"
server_key: "2adf451f83c6e0c0f8169a05b2cb6554"
server_key_iv: "5f7a2d901d9f94fa28a9acdf406af69c"
server_key_version: "encryptionkey@0"
I20260926 16:32:38.408622 17747 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260926 16:32:38.411032 17762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:38.412264 17747 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260926 16:32:38.412485 17747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-2/wal
uuid: "fe17e60ac69141458c23a427d88a8ca9"
format_stamp: "Formatted at 2026-09-26 16:32:38 on dist-test-slave-2zvq"
server_key: "2adf451f83c6e0c0f8169a05b2cb6554"
server_key_iv: "5f7a2d901d9f94fa28a9acdf406af69c"
server_key_version: "encryptionkey@0"
I20260926 16:32:38.412591 17747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:38.465564 17747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:38.468780 17747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:38.468995 17747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:38.469614 17747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:32:38.470516 17747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:38.470571 17747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:38.470638 17747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:38.470670 17747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:38.479929 17747 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.195:39875
I20260926 16:32:38.479960 17875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.195:39875 every 8 connection(s)
I20260926 16:32:38.480863 17747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
I20260926 16:32:38.481583 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 17747
I20260926 16:32:38.481737 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance
I20260926 16:32:38.482041 11299 external_mini_cluster.cc:1428] Setting key 00f56f35a9eccaead23cb02f98e14f7e
Sep 26 16:32:38 dist-test-slave-2zvq krb5kdc[17372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440358, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.195@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM
I20260926 16:32:38.498592 17456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:38.482794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:53279 (local address 127.11.8.254:32843)
0926 16:32:38.482917 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:38.482921 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:38.483854 (+ 933us) server_negotiation.cc:407] Connection header received
0926 16:32:38.485353 (+ 1499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:38.485356 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:38.485401 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:38.485475 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:38.486608 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:38.487193 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:38.488280 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:38.488522 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:38.488610 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:38.494981 (+ 6371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:38.494985 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:38.494988 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:38.495022 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:38.496666 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:38.497133 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:38.497136 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:38.497137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:38.497216 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:38.497509 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:38.497512 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:38.497513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:38.497690 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:38.497769 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:38.498342 (+ 573us) server_negotiation.cc:299] Negotiation successful
0926 16:32:38.498464 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 16:32:38.499382 17878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:38.483237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.11.8.195:53279)
0926 16:32:38.483711 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:38.483767 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:38.485140 (+ 1373us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:38.485610 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:38.485616 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:38.485958 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:38.486468 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:38.486481 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:38.487307 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:38.487309 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:38.488159 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:38.488167 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:38.488649 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:38.488652 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:38.488785 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:38.493160 (+ 4375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:38.493176 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:38.494793 (+ 1617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:38.496766 (+ 1973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:38.496771 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:38.496782 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:38.497041 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:38.497298 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:38.497300 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:38.497302 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:38.497427 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:38.497778 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:38.497783 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:38.498024 (+ 241us) client_negotiation.cc:769] Sending connection context
0926 16:32:38.498179 (+ 155us) client_negotiation.cc:240] Negotiation successful
0926 16:32:38.498502 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":112,"threads_started":1}
I20260926 16:32:38.500451 17876 heartbeater.cc:347] Connected to a master server at 127.11.8.254:32843
I20260926 16:32:38.500695 17876 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:38.501180 17876 heartbeater.cc:511] Master 127.11.8.254:32843 requested a full tablet report, sending...
I20260926 16:32:38.502238 17417 ts_manager.cc:194] Registered new tserver with Master: fe17e60ac69141458c23a427d88a8ca9 (127.11.8.195:39875)
I20260926 16:32:38.503033 17417 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.8.195@KRBTEST.COM'} at 127.11.8.195:53279
I20260926 16:32:38.507948 11299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.11.8.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.8.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.8.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.253.keytab.
Entry for principal oryx/127.11.8.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.253.keytab.
I20260926 16:32:38.537763 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.253:38689
--webserver_interface=127.11.8.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.253.keytab
--principal=oryx/127.11.8.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:40813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:32843,127.11.8.253:38689
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:38.661975 17881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:38.662225 17881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:38.662312 17881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:38.666366 17881 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:32:38.666414 17881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:38.666432 17881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:38.666448 17881 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:32:38.666497 17881 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:32:38.671599 17881 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:40813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:32843,127.11.8.253:38689
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.253.keytab
--principal=oryx/127.11.8.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.253:38689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.8.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17881
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
W20260926 16:32:38.671819 17881 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.11.8.254:32843,127.11.8.253:38689'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260926 16:32:38.672734 17881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:38.673892 17881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:38.682545 17886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:38.682556 17881 server_base.cc:1034] running on GCE node
W20260926 16:32:38.682497 17889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:38.682502 17887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:38.683471 17881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:38.684620 17881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:38.685823 17881 hybrid_clock.cc:648] HybridClock initialized: now 1790440358685804 us; error 31 us; skew 500 ppm
Sep 26 16:32:38 dist-test-slave-2zvq krb5kdc[17372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440358, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:38.688681 17881 init.cc:377] Logged in from keytab as oryx/127.11.8.253@KRBTEST.COM (short username oryx)
I20260926 16:32:38.689489 17881 webserver.cc:460] Webserver started at http://127.11.8.253:34703/ using document root <none> and password file <none>
I20260926 16:32:38.689996 17881 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:38.690053 17881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:38.690212 17881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:38.692524 17881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-1/data/instance:
uuid: "6b832e0a09284005aeec7045d9cffb05"
format_stamp: "Formatted at 2026-09-26 16:32:38 on dist-test-slave-2zvq"
server_key: "9b95772951cc8446b943f9d48ecab6a5"
server_key_iv: "b4c316dae4e35b47bafbaa4c07cbd0ed"
server_key_version: "encryptionkey@0"
I20260926 16:32:38.693096 17881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-1/wal/instance:
uuid: "6b832e0a09284005aeec7045d9cffb05"
format_stamp: "Formatted at 2026-09-26 16:32:38 on dist-test-slave-2zvq"
server_key: "9b95772951cc8446b943f9d48ecab6a5"
server_key_iv: "b4c316dae4e35b47bafbaa4c07cbd0ed"
server_key_version: "encryptionkey@0"
I20260926 16:32:38.697070 17881 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260926 16:32:38.699422 17896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:38.700631 17881 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 16:32:38.700728 17881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-1/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-1/wal
uuid: "6b832e0a09284005aeec7045d9cffb05"
format_stamp: "Formatted at 2026-09-26 16:32:38 on dist-test-slave-2zvq"
server_key: "9b95772951cc8446b943f9d48ecab6a5"
server_key_iv: "b4c316dae4e35b47bafbaa4c07cbd0ed"
server_key_version: "encryptionkey@0"
I20260926 16:32:38.700804 17881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:38.722132 17881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:38.725183 17881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:38.725327 17881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:38.732554 17881 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.253:38689
I20260926 16:32:38.732842 17948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.253:38689 every 8 connection(s)
I20260926 16:32:38.733582 17881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-1/data/info.pb
I20260926 16:32:38.734210 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 17881
I20260926 16:32:38.736824 17949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:38.740552 17949 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 16:32:38 dist-test-slave-2zvq krb5kdc[17372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.8.253@KRBTEST.COM
Sep 26 16:32:38 dist-test-slave-2zvq krb5kdc[17372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440358, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.253@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM
I20260926 16:32:38.749454 17952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:38.735730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36434 (local address 127.11.8.253:38689)
0926 16:32:38.736153 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:38.736167 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:38.736203 (+ 36us) server_negotiation.cc:407] Connection header received
0926 16:32:38.736919 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:38.736943 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:38.737372 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:38.737732 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:38.738634 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:38.739616 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:38.740509 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:38.741022 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:38.741182 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:38.744152 (+ 2970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:38.744160 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:38.744174 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:38.744203 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:38.746131 (+ 1928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:38.746664 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:38.746668 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:38.746673 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:38.746779 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:38.747161 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:38.747164 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:38.747166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:38.747705 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:38.747907 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:38.748149 (+ 242us) server_negotiation.cc:299] Negotiation successful
0926 16:32:38.748436 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":139,"threads_started":1}
I20260926 16:32:38.753340 17456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:38.742391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38468 (local address 127.11.8.254:32843)
0926 16:32:38.742523 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:38.742526 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:38.742892 (+ 366us) server_negotiation.cc:407] Connection header received
0926 16:32:38.743653 (+ 761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:38.743656 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:38.743708 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:38.743807 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:38.744610 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:38.745286 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:38.746558 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:38.746836 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:38.746956 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:38.749392 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:38.749395 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:38.749397 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:38.749427 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:38.751296 (+ 1869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:38.751878 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:38.751882 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:38.751885 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:38.751974 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:38.752277 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:38.752281 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:38.752283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:38.752520 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:38.752638 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:38.753003 (+ 365us) server_negotiation.cc:299] Negotiation successful
0926 16:32:38.753134 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 16:32:38.753384 17953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:38.742364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.0.0.1:38468)
0926 16:32:38.742757 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:38.742812 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:38.742972 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:38.743882 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:38.743887 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:38.744308 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:38.744492 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:38.744499 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:38.745393 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:38.745396 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:38.746452 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:38.746460 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:38.747011 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:38.747015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:38.747138 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:38.747676 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:38.747689 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:38.749258 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:38.751446 (+ 2188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:38.751457 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:38.751463 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:38.751752 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:38.752076 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:38.752080 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:38.752082 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:38.752174 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:38.752636 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:38.752643 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:38.752774 (+ 131us) client_negotiation.cc:769] Sending connection context
0926 16:32:38.752929 (+ 155us) client_negotiation.cc:240] Negotiation successful
0926 16:32:38.753135 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"spinlock_wait_cycles":2432,"thread_start_us":163,"threads_started":1}
I20260926 16:32:38.754814 17949 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 16:32:38 dist-test-slave-2zvq krb5kdc[17372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440358, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.253@KRBTEST.COM for oryx/127.11.8.253@KRBTEST.COM
I20260926 16:32:38.762945 17953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:38.755314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.253:38689 (local address 127.0.0.1:36450)
0926 16:32:38.755444 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:38.755460 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:38.755538 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:38.755786 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:38.755788 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:38.755919 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:38.756034 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:38.756039 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:38.757173 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:38.757175 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:38.757677 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:38.757684 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:38.758138 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:38.758141 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:38.758213 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:38.758758 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:38.758769 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:38.760044 (+ 1275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:38.761669 (+ 1625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:38.761671 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:38.761672 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:38.761869 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:38.762093 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:38.762096 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:38.762097 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:38.762165 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:38.762561 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:38.762563 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:38.762633 (+ 70us) client_negotiation.cc:769] Sending connection context
0926 16:32:38.762716 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 16:32:38.762804 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260926 16:32:38.762979 17952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:38.755505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36450 (local address 127.11.8.253:38689)
0926 16:32:38.755617 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:38.755621 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:38.755631 (+ 10us) server_negotiation.cc:407] Connection header received
0926 16:32:38.755660 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:38.755662 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:38.755694 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:38.755760 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:38.756135 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:38.757068 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:38.757779 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:38.758036 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:38.758116 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:38.760187 (+ 2071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:38.760190 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:38.760192 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:38.760212 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:38.761572 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:38.761940 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:38.761942 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:38.761943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:38.762014 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:38.762237 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:38.762239 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:38.762240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:38.762482 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:38.762560 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:38.762747 (+ 187us) server_negotiation.cc:299] Negotiation successful
0926 16:32:38.762857 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":45}
I20260926 16:32:38.765820 17949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05: Bootstrap starting.
I20260926 16:32:38.768049 17949 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:38.768690 17949 log.cc:824] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05: Log is configured to *not* fsync() on all Append() calls
I20260926 16:32:38.770509 17949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05: No bootstrap required, opened a new log
I20260926 16:32:38.773252 17949 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } }
I20260926 16:32:38.773428 17949 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:38.773461 17949 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b832e0a09284005aeec7045d9cffb05, State: Initialized, Role: FOLLOWER
I20260926 16:32:38.773850 17949 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } }
I20260926 16:32:38.774240 17955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } } }
I20260926 16:32:38.774421 17955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:32:38.774958 17949 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [sys.catalog]: configured and running, proceeding with master startup.
W20260926 16:32:38.779347 17966 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 16:32:38.779439 17966 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 16:32:38.781613 17881 master_runner.cc:428] Detected that this master 6b832e0a09284005aeec7045d9cffb05 is joining an existing cluster
I20260926 16:32:38.781658 17881 master_runner.cc:432] Initiating AddMaster RPC to add 127.11.8.253:38689
I20260926 16:32:38.791853 17456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:38.783844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38474 (local address 127.11.8.254:32843)
0926 16:32:38.784035 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:38.784040 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:38.784054 (+ 14us) server_negotiation.cc:407] Connection header received
0926 16:32:38.784091 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:38.784095 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:38.784147 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:38.784232 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:38.784753 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:38.785574 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:38.786620 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:38.786835 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:38.786952 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:38.788353 (+ 1401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:38.788357 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:38.788359 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:38.788380 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:38.790188 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:38.790657 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:38.790661 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:38.790662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:38.790755 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:38.791020 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:38.791023 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:38.791025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:38.791195 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:38.791301 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:38.791593 (+ 292us) server_negotiation.cc:299] Negotiation successful
0926 16:32:38.791694 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260926 16:32:38.791853 17952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:38.784046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36452 (local address 127.11.8.253:38689)
0926 16:32:38.784154 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:38.784157 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:38.784166 (+ 9us) server_negotiation.cc:407] Connection header received
0926 16:32:38.784191 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:38.784193 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:38.784248 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:38.784318 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:38.784852 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:38.785323 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:38.786444 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:38.786796 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:38.786902 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:38.788238 (+ 1336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:38.788242 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:38.788245 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:38.788271 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:38.790043 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:38.790581 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:38.790587 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:38.790591 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:38.790669 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:38.790957 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:38.790960 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:38.790962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:38.791178 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:38.791277 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:38.791583 (+ 306us) server_negotiation.cc:299] Negotiation successful
0926 16:32:38.791694 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 16:32:38.792366 17972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:38.783499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.0.0.1:38474)
0926 16:32:38.783751 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:38.783767 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:38.783852 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:38.784252 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:38.784254 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:38.784492 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:38.784631 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:38.784638 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:38.785692 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:38.785695 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:38.786513 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:38.786519 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:38.787066 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:38.787071 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:38.787171 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:38.787735 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:38.787749 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:38.788233 (+ 484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:38.790306 (+ 2073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:38.790312 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:38.790316 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:38.790568 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:38.790829 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:38.790831 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:38.790833 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:38.790907 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:38.791337 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:38.791341 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:38.791405 (+ 64us) client_negotiation.cc:769] Sending connection context
0926 16:32:38.791998 (+ 593us) client_negotiation.cc:240] Negotiation successful
0926 16:32:38.792179 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":1,"spinlock_wait_cycles":384,"thread_start_us":91,"threads_started":1}
I20260926 16:32:38.792388 17973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:38.783571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.253:38689 (local address 127.0.0.1:36452)
0926 16:32:38.783858 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:38.783869 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:38.783937 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:38.784414 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:38.784416 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:38.784630 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:38.784750 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:38.784755 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:38.785435 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:38.785438 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:38.786318 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:38.786328 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:38.786923 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:38.786926 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:38.787008 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:38.787551 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:38.787562 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:38.788101 (+ 539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:38.790157 (+ 2056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:38.790159 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:38.790161 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:38.790485 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:38.790773 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:38.790775 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:38.790777 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:38.790860 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:38.791285 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:38.791288 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:38.791405 (+ 117us) client_negotiation.cc:769] Sending connection context
0926 16:32:38.792025 (+ 620us) client_negotiation.cc:240] Negotiation successful
0926 16:32:38.792234 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":108,"spinlock_wait_cycles":34560,"thread_start_us":99,"threads_started":1}
Sep 26 16:32:38 dist-test-slave-2zvq krb5kdc[17372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440357, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.254@KRBTEST.COM for oryx/127.11.8.253@KRBTEST.COM
I20260926 16:32:38.806239 17952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:38.796873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36466 (local address 127.11.8.253:38689)
0926 16:32:38.796988 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:38.796992 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:38.797301 (+ 309us) server_negotiation.cc:407] Connection header received
0926 16:32:38.797481 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:38.797484 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:38.797539 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:38.797616 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:38.798191 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:38.798981 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:38.799822 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:38.800160 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:38.800261 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:38.802682 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:38.802689 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:38.802693 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:38.802720 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:38.804420 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:38.804996 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:38.805002 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:38.805004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:38.805086 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:38.805356 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:38.805359 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:38.805361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:38.805532 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:38.805739 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:38.805982 (+ 243us) server_negotiation.cc:299] Negotiation successful
0926 16:32:38.806086 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 16:32:38.806319 17974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:38.796835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.253:38689 (local address 127.0.0.1:36466)
0926 16:32:38.797181 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:38.797229 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:38.797354 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:38.797664 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:38.797669 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:38.797981 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:38.798090 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:38.798095 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:38.799094 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:38.799097 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:38.799714 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:38.799722 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:38.800266 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:38.800268 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:38.800344 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:38.800860 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:38.800872 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:38.802536 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:38.804559 (+ 2023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:38.804570 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:38.804578 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:38.804895 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:38.805174 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:38.805178 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:38.805180 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:38.805264 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:38.805637 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:38.805645 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:38.805775 (+ 130us) client_negotiation.cc:769] Sending connection context
0926 16:32:38.805932 (+ 157us) client_negotiation.cc:240] Negotiation successful
0926 16:32:38.806130 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":117,"threads_started":1}
I20260926 16:32:38.807871 17417 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.11.8.253:38689: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "6b832e0a09284005aeec7045d9cffb05"
member_type: NON_VOTER
last_known_addr {
host: "127.11.8.253"
port: 38689
}
attrs {
promote: true
}
}
dest_uuid: "d1aa72ae6508448486ff84262f3af952"
cas_config_opid_index: -1
I20260926 16:32:38.808552 17417 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: NON_VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: true } }
I20260926 16:32:38.810546 17978 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 6b832e0a09284005aeec7045d9cffb05 (127.11.8.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: NON_VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: true } } }
I20260926 16:32:38.810873 17923 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:38.812325 17923 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [term 1 FOLLOWER]: Refusing update from remote peer d1aa72ae6508448486ff84262f3af952: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260926 16:32:38.812364 17978 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.11.8.253:38689
I20260926 16:32:38.812808 17881 master.cc:537] Master@127.11.8.253:38689 shutting down...
I20260926 16:32:38.813593 17978 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: NON_VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
WARNING: no policy specified for oryx/127.11.8.252@KRBTEST.COM; defaulting to no policy
I20260926 16:32:38.815299 17977 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d1aa72ae6508448486ff84262f3af952" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: NON_VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: true } } }
I20260926 16:32:38.815477 17977 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [sys.catalog]: This master's current role is: LEADER
W20260926 16:32:38.815516 17401 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260926 16:32:38.816376 17401 consensus_peers.cc:487] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 -> Peer 6b832e0a09284005aeec7045d9cffb05 (127.11.8.253:38689): Couldn't send request to peer 6b832e0a09284005aeec7045d9cffb05. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.8.252@KRBTEST.COM" created.
I20260926 16:32:38.817646 17881 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:32:38.817929 17881 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:32:38.818061 17881 tablet_replica.cc:330] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05: stopping tablet replica
I20260926 16:32:38.822353 17881 master.cc:552] Master@127.11.8.253:38689 shutdown complete.
I20260926 16:32:38.822403 17881 master_runner.cc:305] Clearing existing system tablet
I20260926 16:32:38.822675 17881 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:38.822737 17881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:38.824222 17881 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260926 16:32:38.825160 17984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:38.825407 17881 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:38.825554 17881 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 16:32:38.825600 17881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-1/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-1/wal
uuid: "6b832e0a09284005aeec7045d9cffb05"
format_stamp: "Formatted at 2026-09-26 16:32:38 on dist-test-slave-2zvq"
server_key: "9b95772951cc8446b943f9d48ecab6a5"
server_key_iv: "b4c316dae4e35b47bafbaa4c07cbd0ed"
server_key_version: "encryptionkey@0"
I20260926 16:32:38.827612 17881 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:32:38.830498 17881 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260926 16:32:38.830595 17881 log.cc:1196] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260926 16:32:38.830899 17881 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05: Deleting consensus metadata
I20260926 16:32:38.831108 17881 master_runner.cc:315] Copying system tablet from 127.11.8.254:32843
I20260926 16:32:38.832279 17881 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05: tablet copy: Beginning tablet copy session from remote peer at address 127.11.8.254:32843
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.8.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.252.keytab.
Entry for principal oryx/127.11.8.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.252.keytab.
I20260926 16:32:38.837127 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.252:40855
--webserver_interface=127.11.8.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.252.keytab
--principal=oryx/127.11.8.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:40813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:32843,127.11.8.253:38689,127.11.8.252:40855
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 16:32:38.841220 17456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:38.832962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38478 (local address 127.11.8.254:32843)
0926 16:32:38.833112 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:38.833122 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:38.833201 (+ 79us) server_negotiation.cc:407] Connection header received
0926 16:32:38.833372 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:38.833376 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:38.833428 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:38.833530 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:38.833990 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:38.834980 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:38.835839 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:38.836229 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:38.836333 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:38.837764 (+ 1431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:38.837768 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:38.837771 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:38.837798 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:38.839516 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:38.840012 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:38.840016 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:38.840018 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:38.840099 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:38.840384 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:38.840388 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:38.840390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:38.840568 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:38.840669 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:38.840939 (+ 270us) server_negotiation.cc:299] Negotiation successful
0926 16:32:38.841057 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 16:32:38.841234 17990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:38.832853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.0.0.1:38478)
0926 16:32:38.833120 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:38.833135 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:38.833267 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:38.833539 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:38.833540 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:38.833753 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:38.833874 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:38.833881 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:38.835098 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:38.835101 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:38.835729 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:38.835738 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:38.836467 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:38.836471 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:38.836567 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:38.837165 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:38.837180 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:38.837593 (+ 413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:38.839641 (+ 2048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:38.839644 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:38.839646 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:38.839911 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:38.840192 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:38.840196 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:38.840198 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:38.840285 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:38.840670 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:38.840674 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:38.840773 (+ 99us) client_negotiation.cc:769] Sending connection context
0926 16:32:38.840876 (+ 103us) client_negotiation.cc:240] Negotiation successful
0926 16:32:38.841021 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":107,"threads_started":1}
I20260926 16:32:38.842038 17437 tablet_copy_service.cc:140] P d1aa72ae6508448486ff84262f3af952: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 6b832e0a09284005aeec7045d9cffb05 ({username='oryx', principal='oryx/127.11.8.253@KRBTEST.COM'} at 127.0.0.1:38478)
I20260926 16:32:38.842221 17437 tablet_copy_service.cc:161] P d1aa72ae6508448486ff84262f3af952: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 6b832e0a09284005aeec7045d9cffb05 at {username='oryx', principal='oryx/127.11.8.253@KRBTEST.COM'} at 127.0.0.1:38478: session id = 6b832e0a09284005aeec7045d9cffb05-00000000000000000000000000000000
I20260926 16:32:38.843693 17437 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952: Tablet Copy: opened 0 blocks and 1 log segments
I20260926 16:32:38.845118 17881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260926 16:32:38.847004 17881 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05: tablet copy: Starting download of 0 data blocks...
I20260926 16:32:38.847205 17881 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05: tablet copy: Starting download of 1 WAL segments...
I20260926 16:32:38.848769 17881 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260926 16:32:38.850284 17437 tablet_copy_service.cc:342] P d1aa72ae6508448486ff84262f3af952: Request end of tablet copy session 6b832e0a09284005aeec7045d9cffb05-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.11.8.253@KRBTEST.COM'} at 127.0.0.1:38478
I20260926 16:32:38.850414 17437 tablet_copy_service.cc:433] P d1aa72ae6508448486ff84262f3af952: ending tablet copy session 6b832e0a09284005aeec7045d9cffb05-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 6b832e0a09284005aeec7045d9cffb05
I20260926 16:32:38.856740 17881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:38.858310 17992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:38.858330 17995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:38.858634 17881 server_base.cc:1034] running on GCE node
W20260926 16:32:38.861101 17993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:38.861347 17881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:38.861655 17881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:38.862787 17881 hybrid_clock.cc:648] HybridClock initialized: now 1790440358862795 us; error 41 us; skew 500 ppm
Sep 26 16:32:38 dist-test-slave-2zvq krb5kdc[17372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440358, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:38.864182 17881 init.cc:377] Logged in from keytab as oryx/127.11.8.253@KRBTEST.COM (short username oryx)
I20260926 16:32:38.864631 17881 webserver.cc:460] Webserver started at http://127.11.8.253:43779/ using document root <none> and password file <none>
I20260926 16:32:38.864817 17881 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:38.864871 17881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:38.865883 17881 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 16:32:38.866806 18002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:38.867039 17881 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260926 16:32:38.867129 17881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-1/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-1/wal
uuid: "6b832e0a09284005aeec7045d9cffb05"
format_stamp: "Formatted at 2026-09-26 16:32:38 on dist-test-slave-2zvq"
server_key: "9b95772951cc8446b943f9d48ecab6a5"
server_key_iv: "b4c316dae4e35b47bafbaa4c07cbd0ed"
server_key_version: "encryptionkey@0"
I20260926 16:32:38.867209 17881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:38.881319 17881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:38.883543 17881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:38.888293 17881 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.253:38689
I20260926 16:32:38.888473 18054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.253:38689 every 8 connection(s)
I20260926 16:32:38.888948 17881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-1/data/info.pb
I20260926 16:32:38.890453 18055 sys_catalog.cc:263] Verifying existing consensus state
I20260926 16:32:38.891304 18055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05: Bootstrap starting.
I20260926 16:32:38.899133 18055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 16:32:38.899412 18055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05: Bootstrap complete.
I20260926 16:32:38.899861 18055 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: NON_VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: true } }
I20260926 16:32:38.899999 18055 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6b832e0a09284005aeec7045d9cffb05, State: Initialized, Role: LEARNER
I20260926 16:32:38.900112 18055 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: NON_VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: true } }
I20260926 16:32:38.900431 18058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: NON_VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: true } } }
I20260926 16:32:38.900593 18058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [sys.catalog]: This master's current role is: LEARNER
I20260926 16:32:38.900599 18055 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:32:38.904212 18069 catalog_manager.cc:1259] Loaded cluster ID: 2d3a2295b7af482ea4f34e28e17b0044
I20260926 16:32:38.904282 18069 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05: loading cluster ID for follower catalog manager: success
I20260926 16:32:38.905721 18069 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05: acquiring CA information for follower catalog manager: success
I20260926 16:32:38.906831 18069 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260926 16:32:38.991549 17991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:38.991928 17991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:38.991992 17991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:38.996080 17991 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:32:38.996140 17991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:38.996160 17991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:38.996176 17991 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:32:38.996191 17991 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:32:39.001581 17991 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:40813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:32843,127.11.8.253:38689,127.11.8.252:40855
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.252.keytab
--principal=oryx/127.11.8.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.252:40855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.8.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:39.002825 17991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:39.003981 17991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:39.012404 18075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:39.012529 18077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:39.012444 18074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:39.012434 17991 server_base.cc:1034] running on GCE node
I20260926 16:32:39.013155 17991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:39.014258 17991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:39.015511 17991 hybrid_clock.cc:648] HybridClock initialized: now 1790440359015501 us; error 30 us; skew 500 ppm
Sep 26 16:32:39 dist-test-slave-2zvq krb5kdc[17372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440359, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:39.018126 17991 init.cc:377] Logged in from keytab as oryx/127.11.8.252@KRBTEST.COM (short username oryx)
I20260926 16:32:39.018985 17991 webserver.cc:460] Webserver started at http://127.11.8.252:42165/ using document root <none> and password file <none>
I20260926 16:32:39.019513 17991 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:39.019572 17991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:39.019798 17991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:39.021786 17991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-2/data/instance:
uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9"
format_stamp: "Formatted at 2026-09-26 16:32:39 on dist-test-slave-2zvq"
server_key: "f3206075ea1f557c703521aa1560e763"
server_key_iv: "41783b48a75524e6b280e8153da87e89"
server_key_version: "encryptionkey@0"
I20260926 16:32:39.022485 17991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-2/wal/instance:
uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9"
format_stamp: "Formatted at 2026-09-26 16:32:39 on dist-test-slave-2zvq"
server_key: "f3206075ea1f557c703521aa1560e763"
server_key_iv: "41783b48a75524e6b280e8153da87e89"
server_key_version: "encryptionkey@0"
I20260926 16:32:39.026230 17991 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260926 16:32:39.026489 17608 heartbeater.cc:503] Master 127.11.8.254:32843 was elected leader, sending a full tablet report...
I20260926 16:32:39.028833 18084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:39.030200 17991 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:32:39.030352 17991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-2/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-2/wal
uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9"
format_stamp: "Formatted at 2026-09-26 16:32:39 on dist-test-slave-2zvq"
server_key: "f3206075ea1f557c703521aa1560e763"
server_key_iv: "41783b48a75524e6b280e8153da87e89"
server_key_version: "encryptionkey@0"
I20260926 16:32:39.030445 17991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:39.069655 17991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:39.072960 17991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:39.073124 17991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:39.080363 17991 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.252:40855
I20260926 16:32:39.080648 18136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.252:40855 every 8 connection(s)
I20260926 16:32:39.081398 17991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-2/data/info.pb
I20260926 16:32:39.084218 18137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:39.084962 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 17991
I20260926 16:32:39.089250 18137 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 16:32:39 dist-test-slave-2zvq krb5kdc[17372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.8.252@KRBTEST.COM
Sep 26 16:32:39 dist-test-slave-2zvq krb5kdc[17372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440359, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.252@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM
I20260926 16:32:39.101112 18139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.086485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53936 (local address 127.11.8.252:40855)
0926 16:32:39.086870 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:39.086888 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:39.086929 (+ 41us) server_negotiation.cc:407] Connection header received
0926 16:32:39.087681 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:39.087704 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:39.088109 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:39.088479 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:39.089309 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.090750 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.091958 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.092628 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.092739 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.095972 (+ 3233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:39.095977 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:39.095986 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:39.096033 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:39.097848 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:39.098306 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:39.098315 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:39.098322 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:39.098455 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:39.098724 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:39.098729 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:39.098731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:39.099281 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:39.099496 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:39.099787 (+ 291us) server_negotiation.cc:299] Negotiation successful
0926 16:32:39.100115 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":102,"threads_started":1}
I20260926 16:32:39.101440 17456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.091272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38494 (local address 127.11.8.254:32843)
0926 16:32:39.091393 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:39.091397 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:39.092053 (+ 656us) server_negotiation.cc:407] Connection header received
0926 16:32:39.092190 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:39.092193 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:39.092242 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:39.092328 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:39.093040 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.093696 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.094973 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.095350 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.095446 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.097973 (+ 2527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:39.097977 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:39.097980 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:39.098005 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:39.099665 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:39.100214 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:39.100218 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:39.100220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:39.100304 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:39.100569 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:39.100573 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:39.100574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:39.100773 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:39.100879 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:39.101178 (+ 299us) server_negotiation.cc:299] Negotiation successful
0926 16:32:39.101293 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 16:32:39.101563 18140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.091110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.0.0.1:38494)
0926 16:32:39.091914 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:39.091969 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:39.092091 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:39.092378 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:39.092383 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:39.092752 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:39.092907 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.092915 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.093833 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.093839 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.094829 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.094839 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.095499 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.095503 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.095630 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.096151 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:39.096170 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:39.097834 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:39.099790 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:39.099798 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:39.099805 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:39.100117 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:39.100392 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:39.100394 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:39.100396 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:39.100485 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:39.100881 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:39.100886 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:39.100978 (+ 92us) client_negotiation.cc:769] Sending connection context
0926 16:32:39.101088 (+ 110us) client_negotiation.cc:240] Negotiation successful
0926 16:32:39.101257 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":77,"threads_started":1}
I20260926 16:32:39.103129 18137 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 16:32:39 dist-test-slave-2zvq krb5kdc[17372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440359, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.252@KRBTEST.COM for oryx/127.11.8.253@KRBTEST.COM
I20260926 16:32:39.110989 18140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.103673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.253:38689 (local address 127.0.0.1:36480)
0926 16:32:39.103806 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:39.103821 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:39.103895 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:39.104377 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:39.104379 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:39.104500 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:39.104580 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.104585 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.105460 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.105463 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.105964 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.105971 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.106657 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.106660 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.106730 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.107160 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:39.107174 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:39.108277 (+ 1103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:39.109736 (+ 1459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:39.109738 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:39.109739 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:39.109911 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:39.110164 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:39.110166 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:39.110167 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:39.110233 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:39.110616 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:39.110618 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:39.110686 (+ 68us) client_negotiation.cc:769] Sending connection context
0926 16:32:39.110766 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 16:32:39.110857 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260926 16:32:39.111037 18141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.103857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36480 (local address 127.11.8.253:38689)
0926 16:32:39.104165 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:39.104169 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:39.104182 (+ 13us) server_negotiation.cc:407] Connection header received
0926 16:32:39.104218 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:39.104221 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:39.104258 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:39.104341 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:39.104680 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.105338 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.106229 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.106552 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.106641 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.108396 (+ 1755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:39.108399 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:39.108401 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:39.108423 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:39.109645 (+ 1222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:39.110010 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:39.110013 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:39.110014 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:39.110087 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:39.110360 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:39.110365 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:39.110371 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:39.110531 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:39.110607 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:39.110826 (+ 219us) server_negotiation.cc:299] Negotiation successful
0926 16:32:39.110917 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":108,"threads_started":1}
I20260926 16:32:39.111860 18137 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.252" port: 40855 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 16:32:39 dist-test-slave-2zvq krb5kdc[17372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440359, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.252@KRBTEST.COM for oryx/127.11.8.252@KRBTEST.COM
I20260926 16:32:39.119529 18140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.112351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:40855 (local address 127.0.0.1:53948)
0926 16:32:39.112450 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:39.112464 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:39.112519 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:39.112716 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:39.112718 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:39.112833 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:39.112894 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.112898 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.113922 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.113924 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.114507 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.114515 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.114938 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.114940 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.115012 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.115479 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:39.115490 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:39.116671 (+ 1181us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:39.118386 (+ 1715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:39.118389 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:39.118390 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:39.118559 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:39.118781 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:39.118783 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:39.118784 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:39.118858 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:39.119172 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:39.119174 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:39.119241 (+ 67us) client_negotiation.cc:769] Sending connection context
0926 16:32:39.119321 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 16:32:39.119408 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260926 16:32:39.119660 18139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.112449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53948 (local address 127.11.8.252:40855)
0926 16:32:39.112550 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:39.112554 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:39.112573 (+ 19us) server_negotiation.cc:407] Connection header received
0926 16:32:39.112600 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:39.112602 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:39.112632 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:39.112693 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:39.112999 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.113832 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.114607 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.114843 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.114930 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.116785 (+ 1855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:39.116788 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:39.116790 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:39.116835 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:39.118242 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:39.118636 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:39.118638 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:39.118639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:39.118706 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:39.118932 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:39.118934 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:39.118935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:39.119091 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:39.119165 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:39.119408 (+ 243us) server_negotiation.cc:299] Negotiation successful
0926 16:32:39.119499 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 16:32:39.122756 18137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9: Bootstrap starting.
I20260926 16:32:39.125067 18137 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:39.125785 18137 log.cc:824] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9: Log is configured to *not* fsync() on all Append() calls
I20260926 16:32:39.127760 18137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9: No bootstrap required, opened a new log
I20260926 16:32:39.130900 18137 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 40855 } }
I20260926 16:32:39.131125 18137 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:39.131203 18137 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff1d5a9204f14f58bdac74c8ee80c8c9, State: Initialized, Role: FOLLOWER
I20260926 16:32:39.131611 18137 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 40855 } }
I20260926 16:32:39.132054 18143 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 40855 } } }
I20260926 16:32:39.132166 18143 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:32:39.132743 18137 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [sys.catalog]: configured and running, proceeding with master startup.
W20260926 16:32:39.138016 18154 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 16:32:39.138092 18154 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 16:32:39.140746 17991 master_runner.cc:428] Detected that this master ff1d5a9204f14f58bdac74c8ee80c8c9 is joining an existing cluster
I20260926 16:32:39.140820 17991 master_runner.cc:432] Initiating AddMaster RPC to add 127.11.8.252:40855
I20260926 16:32:39.151741 17456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.142710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38500 (local address 127.11.8.254:32843)
0926 16:32:39.142833 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:39.142840 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:39.144070 (+ 1230us) server_negotiation.cc:407] Connection header received
0926 16:32:39.144242 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:39.144245 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:39.144292 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:39.144343 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:39.144942 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.145752 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.147032 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.147365 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.147456 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.148738 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:39.148741 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:39.148743 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:39.148765 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:39.150208 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:39.150691 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:39.150694 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:39.150695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:39.150766 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:39.151030 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:39.151032 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:39.151033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:39.151190 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:39.151263 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:39.151535 (+ 272us) server_negotiation.cc:299] Negotiation successful
0926 16:32:39.151625 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 16:32:39.151805 18162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.142821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.0.0.1:38500)
0926 16:32:39.143990 (+ 1169us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:39.144003 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:39.144135 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:39.144494 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:39.144497 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:39.144694 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:39.144800 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.144807 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.145892 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.145895 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.146892 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.146903 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.147604 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.147607 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.147697 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.148179 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:39.148193 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:39.148609 (+ 416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:39.150352 (+ 1743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:39.150355 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:39.150358 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:39.150604 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:39.150852 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:39.150855 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:39.150857 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:39.150946 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:39.151279 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:39.151282 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:39.151375 (+ 93us) client_negotiation.cc:769] Sending connection context
0926 16:32:39.151489 (+ 114us) client_negotiation.cc:240] Negotiation successful
0926 16:32:39.151622 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1114,"thread_start_us":87,"threads_started":1}
I20260926 16:32:39.154513 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 17479
I20260926 16:32:39.162026 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.193:42459
--local_ip_for_outbound_sockets=127.11.8.193
--tserver_master_addrs=127.11.8.254:32843
--webserver_port=37679
--webserver_interface=127.11.8.193
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.193.keytab
--principal=oryx/127.11.8.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:40813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.8.254:32843,127.11.8.253:38689
--tserver_master_addrs=127.11.8.254:32843,127.11.8.253:38689,127.11.8.252:40855 with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 16:32:39 dist-test-slave-2zvq krb5kdc[17372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440357, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.254@KRBTEST.COM for oryx/127.11.8.252@KRBTEST.COM
I20260926 16:32:39.170307 17974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.159915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:40855 (local address 127.0.0.1:53964)
0926 16:32:39.160113 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:39.160128 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:39.160208 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:39.160498 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:39.160502 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:39.160702 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:39.160788 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.160793 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.161556 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.161560 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.162453 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.162468 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.163068 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.163074 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.163201 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.163718 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:39.163732 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:39.164929 (+ 1197us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:39.168718 (+ 3789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:39.168721 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:39.168723 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:39.168966 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:39.169247 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:39.169250 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:39.169252 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:39.169330 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:39.169777 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:39.169782 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:39.169876 (+ 94us) client_negotiation.cc:769] Sending connection context
0926 16:32:39.169973 (+ 97us) client_negotiation.cc:240] Negotiation successful
0926 16:32:39.170110 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116}
I20260926 16:32:39.171044 18163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.159960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53964 (local address 127.11.8.252:40855)
0926 16:32:39.160222 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:39.160232 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:39.160256 (+ 24us) server_negotiation.cc:407] Connection header received
0926 16:32:39.160335 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:39.160337 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:39.160380 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:39.160481 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:39.160902 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.161439 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.162583 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.162939 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.163045 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.165078 (+ 2033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:39.165082 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:39.165085 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:39.165110 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:39.168566 (+ 3456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:39.169074 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:39.169077 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:39.169079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:39.169153 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:39.169437 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:39.169440 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:39.169442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:39.169648 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:39.169733 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:39.170761 (+ 1028us) server_negotiation.cc:299] Negotiation successful
0926 16:32:39.170877 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":95,"threads_started":1}
I20260926 16:32:39.172173 17417 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.11.8.252:40855: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9"
member_type: NON_VOTER
last_known_addr {
host: "127.11.8.252"
port: 40855
}
attrs {
promote: true
}
}
dest_uuid: "d1aa72ae6508448486ff84262f3af952"
cas_config_opid_index: 5
I20260926 16:32:39.172353 18160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.142750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.253:38689 (local address 127.0.0.1:36486)
0926 16:32:39.143333 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:39.143348 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:39.143497 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:39.143725 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:39.143729 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:39.143961 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:39.144111 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.144117 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.145199 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.145203 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.146215 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.146224 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.146914 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.146916 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.146995 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.167585 (+ 20590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:39.167611 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:39.168185 (+ 574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:39.170529 (+ 2344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:39.170535 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:39.170539 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:39.170802 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:39.171147 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:39.171154 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:39.171158 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:39.171256 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:39.171793 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:39.171799 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:39.171908 (+ 109us) client_negotiation.cc:769] Sending connection context
0926 16:32:39.172019 (+ 111us) client_negotiation.cc:240] Negotiation successful
0926 16:32:39.172177 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"mutex_wait_us":65,"spinlock_wait_cycles":768,"thread_start_us":130,"threads_started":1}
I20260926 16:32:39.172799 18141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.143149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36486 (local address 127.11.8.253:38689)
0926 16:32:39.143268 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:39.143272 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:39.143457 (+ 185us) server_negotiation.cc:407] Connection header received
0926 16:32:39.143583 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:39.143585 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:39.143620 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:39.143710 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:39.144252 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.145079 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.146370 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.146722 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.146783 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.168379 (+ 21596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:39.168385 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:39.168391 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:39.168431 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:39.170383 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:39.170933 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:39.170939 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:39.170942 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:39.171034 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:39.171368 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:39.171373 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:39.171377 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:39.171640 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:39.171743 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:39.172538 (+ 795us) server_negotiation.cc:299] Negotiation successful
0926 16:32:39.172664 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 16:32:39.172829 17417 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: NON_VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: true } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: NON_VOTER last_known_addr { host: "127.11.8.252" port: 40855 } attrs { promote: true } }
I20260926 16:32:39.174615 17977 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER ff1d5a9204f14f58bdac74c8ee80c8c9 (127.11.8.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: NON_VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: true } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: NON_VOTER last_known_addr { host: "127.11.8.252" port: 40855 } attrs { promote: true } } }
I20260926 16:32:39.175235 18161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.142788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:40855 (local address 127.0.0.1:53960)
0926 16:32:39.143682 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:39.143694 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:39.143792 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:39.144094 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:39.144099 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:39.144306 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:39.144406 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.144412 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.145304 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.145307 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.146215 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.146225 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.146881 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.146884 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.146986 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.147569 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:39.147583 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:39.147971 (+ 388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:39.172961 (+ 24990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:39.172964 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:39.172966 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:39.173187 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:39.173972 (+ 785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:39.173975 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:39.173977 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:39.174053 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:39.174557 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:39.174560 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:39.174634 (+ 74us) client_negotiation.cc:769] Sending connection context
0926 16:32:39.174732 (+ 98us) client_negotiation.cc:240] Negotiation successful
0926 16:32:39.175068 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":839,"mutex_wait_us":33,"thread_start_us":102,"threads_started":1}
I20260926 16:32:39.175658 18139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.143112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53960 (local address 127.11.8.252:40855)
0926 16:32:39.143219 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:39.143223 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:39.143749 (+ 526us) server_negotiation.cc:407] Connection header received
0926 16:32:39.143863 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:39.143866 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:39.143906 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:39.144007 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:39.144512 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.145199 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.146363 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.146665 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.146730 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.148105 (+ 1375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:39.148111 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:39.148116 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:39.148140 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:39.172360 (+ 24220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:39.173788 (+ 1428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:39.173791 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:39.173793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:39.173874 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:39.174143 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:39.174146 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:39.174147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:39.174435 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:39.174524 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:39.174985 (+ 461us) server_negotiation.cc:299] Negotiation successful
0926 16:32:39.175532 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":436,"server-negotiator.queue_time_us":45}
I20260926 16:32:39.176401 17977 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.11.8.252:40855
I20260926 16:32:39.176601 17977 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d1aa72ae6508448486ff84262f3af952" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: NON_VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: true } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: NON_VOTER last_known_addr { host: "127.11.8.252" port: 40855 } attrs { promote: true } } }
I20260926 16:32:39.176719 17977 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:39.178304 18111 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:39.179633 18111 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [term 1 FOLLOWER]: Refusing update from remote peer d1aa72ae6508448486ff84262f3af952: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260926 16:32:39.181538 17979 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: NON_VOTER last_known_addr { host: "127.11.8.252" port: 40855 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:32:39.183007 17974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.177047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.253:38689 (local address 127.0.0.1:36498)
0926 16:32:39.178396 (+ 1349us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:39.178410 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:39.178487 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:39.178884 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:39.178887 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:39.178905 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:39.178998 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.179004 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.180087 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.180090 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.181372 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.181382 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.182604 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.182608 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.182704 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.182751 (+ 47us) client_negotiation.cc:769] Sending connection context
0926 16:32:39.182825 (+ 74us) client_negotiation.cc:240] Negotiation successful
0926 16:32:39.182891 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1286}
I20260926 16:32:39.183393 18141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.177901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36498 (local address 127.11.8.253:38689)
0926 16:32:39.178221 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:39.178225 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:39.178562 (+ 337us) server_negotiation.cc:407] Connection header received
0926 16:32:39.178601 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:39.178604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:39.178656 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:39.178735 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:39.179152 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.179958 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.181505 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.182178 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.182309 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.182471 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:39.183219 (+ 748us) server_negotiation.cc:299] Negotiation successful
0926 16:32:39.183291 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216}
I20260926 16:32:39.184842 18029 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [term 1 LEARNER]: Refusing update from remote peer d1aa72ae6508448486ff84262f3af952: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260926 16:32:39.185300 17991 master.cc:537] Master@127.11.8.252:40855 shutting down...
I20260926 16:32:39.185283 17979 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: NON_VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260926 16:32:39.187574 18111 raft_consensus.cc:858] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 40855 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: NON_VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: NON_VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: true } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: NON_VOTER last_known_addr { host: "127.11.8.252" port: 40855 } attrs { promote: true } }
W20260926 16:32:39.187749 18111 raft_consensus.cc:1565] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 40855 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: NON_VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: NON_VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: true } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: NON_VOTER last_known_addr { host: "127.11.8.252" port: 40855 } attrs { promote: true } }
I20260926 16:32:39.188352 18029 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER ff1d5a9204f14f58bdac74c8ee80c8c9 (127.11.8.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: NON_VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: true } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: NON_VOTER last_known_addr { host: "127.11.8.252" port: 40855 } attrs { promote: true } } }
I20260926 16:32:39.189216 18111 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 6b832e0a09284005aeec7045d9cffb05 (127.11.8.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: NON_VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: true } } }
I20260926 16:32:39.190140 18058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1aa72ae6508448486ff84262f3af952. Latest consensus state: current_term: 1 leader_uuid: "d1aa72ae6508448486ff84262f3af952" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: NON_VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: true } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: NON_VOTER last_known_addr { host: "127.11.8.252" port: 40855 } attrs { promote: true } } }
I20260926 16:32:39.190279 18058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [sys.catalog]: This master's current role is: LEARNER
I20260926 16:32:39.190366 18058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d1aa72ae6508448486ff84262f3af952" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: NON_VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: true } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: NON_VOTER last_known_addr { host: "127.11.8.252" port: 40855 } attrs { promote: true } } }
I20260926 16:32:39.190438 18058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [sys.catalog]: This master's current role is: LEARNER
I20260926 16:32:39.191538 18143 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1aa72ae6508448486ff84262f3af952. Latest consensus state: current_term: 1 leader_uuid: "d1aa72ae6508448486ff84262f3af952" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: NON_VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: true } } }
I20260926 16:32:39.191663 18143 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260926 16:32:39.191748 18143 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d1aa72ae6508448486ff84262f3af952" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: NON_VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: true } } }
I20260926 16:32:39.191802 18143 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260926 16:32:39.191859 18143 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d1aa72ae6508448486ff84262f3af952" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: NON_VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: true } } }
I20260926 16:32:39.191907 18143 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260926 16:32:39.193008 17979 raft_consensus.cc:1050] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952: attempting to promote NON_VOTER 6b832e0a09284005aeec7045d9cffb05 to VOTER
I20260926 16:32:39.193456 17979 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: false } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: NON_VOTER last_known_addr { host: "127.11.8.252" port: 40855 } attrs { promote: true } }
I20260926 16:32:39.194813 18029 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [term 1 LEARNER]: Refusing update from remote peer d1aa72ae6508448486ff84262f3af952: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260926 16:32:39.195164 17978 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260926 16:32:39.195415 17403 consensus_peers.cc:487] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 -> Peer ff1d5a9204f14f58bdac74c8ee80c8c9 (127.11.8.252:40855): Couldn't send request to peer ff1d5a9204f14f58bdac74c8ee80c8c9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260926 16:32:39.196696 17977 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 6b832e0a09284005aeec7045d9cffb05 (127.11.8.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: false } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: NON_VOTER last_known_addr { host: "127.11.8.252" port: 40855 } attrs { promote: true } } }
I20260926 16:32:39.197103 18029 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 6b832e0a09284005aeec7045d9cffb05 (127.11.8.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: false } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: NON_VOTER last_known_addr { host: "127.11.8.252" port: 40855 } attrs { promote: true } } }
I20260926 16:32:39.198094 17977 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d1aa72ae6508448486ff84262f3af952" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: false } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: NON_VOTER last_known_addr { host: "127.11.8.252" port: 40855 } attrs { promote: true } } }
I20260926 16:32:39.198190 17977 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:39.198325 17977 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d1aa72ae6508448486ff84262f3af952" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: false } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: NON_VOTER last_known_addr { host: "127.11.8.252" port: 40855 } attrs { promote: true } } }
I20260926 16:32:39.198393 17977 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:39.198601 18169 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d1aa72ae6508448486ff84262f3af952" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: false } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: NON_VOTER last_known_addr { host: "127.11.8.252" port: 40855 } attrs { promote: true } } }
I20260926 16:32:39.198721 18169 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:32:39.201083 18165 mvcc.cc:205] Tried to move back new op lower bound from 7333643705443119104 to 7333643705251246080. Current Snapshot: MvccSnapshot[applied={T|T < 7333643705283530752}]
I20260926 16:32:39.204306 17991 raft_consensus.cc:2229] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260926 16:32:39.204586 17991 raft_consensus.cc:2258] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260926 16:32:39.204692 17991 tablet_replica.cc:330] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9: stopping tablet replica
I20260926 16:32:39.209510 17991 master.cc:552] Master@127.11.8.252:40855 shutdown complete.
I20260926 16:32:39.209576 17991 master_runner.cc:305] Clearing existing system tablet
I20260926 16:32:39.209782 17991 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:39.209848 17991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:39.210947 17991 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260926 16:32:39.211737 18175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:39.211959 17991 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:39.212106 17991 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 16:32:39.212142 17991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-2/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-2/wal
uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9"
format_stamp: "Formatted at 2026-09-26 16:32:39 on dist-test-slave-2zvq"
server_key: "f3206075ea1f557c703521aa1560e763"
server_key_iv: "41783b48a75524e6b280e8153da87e89"
server_key_version: "encryptionkey@0"
I20260926 16:32:39.213423 17991 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:32:39.215454 17991 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260926 16:32:39.215548 17991 log.cc:1196] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260926 16:32:39.215883 17991 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9: Deleting consensus metadata
I20260926 16:32:39.216065 17991 master_runner.cc:315] Copying system tablet from 127.11.8.254:32843
I20260926 16:32:39.217154 17991 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9: tablet copy: Beginning tablet copy session from remote peer at address 127.11.8.254:32843
I20260926 16:32:39.224615 17456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.217685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38504 (local address 127.11.8.254:32843)
0926 16:32:39.217811 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:39.217818 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:39.217992 (+ 174us) server_negotiation.cc:407] Connection header received
0926 16:32:39.218133 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:39.218136 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:39.218189 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:39.218341 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:39.218766 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.219298 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.220118 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.220347 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.220462 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.221506 (+ 1044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:39.221510 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:39.221513 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:39.221536 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:39.223003 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:39.223443 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:39.223446 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:39.223448 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:39.223524 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:39.223792 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:39.223797 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:39.223798 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:39.223988 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:39.224096 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:39.224349 (+ 253us) server_negotiation.cc:299] Negotiation successful
0926 16:32:39.224470 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 16:32:39.224618 18181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.217615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.0.0.1:38504)
0926 16:32:39.217903 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:39.217914 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:39.218039 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:39.218323 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:39.218328 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:39.218557 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:39.218650 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.218656 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.219404 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.219410 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.219992 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.220000 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.220454 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.220458 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.220552 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.221057 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:39.221068 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:39.221368 (+ 300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:39.223132 (+ 1764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:39.223136 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:39.223139 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:39.223339 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:39.223613 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:39.223618 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:39.223620 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:39.223704 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:39.224085 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:39.224088 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:39.224188 (+ 100us) client_negotiation.cc:769] Sending connection context
0926 16:32:39.224293 (+ 105us) client_negotiation.cc:240] Negotiation successful
0926 16:32:39.224431 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":81,"threads_started":1}
I20260926 16:32:39.225308 17437 tablet_copy_service.cc:140] P d1aa72ae6508448486ff84262f3af952: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer ff1d5a9204f14f58bdac74c8ee80c8c9 ({username='oryx', principal='oryx/127.11.8.252@KRBTEST.COM'} at 127.0.0.1:38504)
I20260926 16:32:39.225419 17437 tablet_copy_service.cc:161] P d1aa72ae6508448486ff84262f3af952: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer ff1d5a9204f14f58bdac74c8ee80c8c9 at {username='oryx', principal='oryx/127.11.8.252@KRBTEST.COM'} at 127.0.0.1:38504: session id = ff1d5a9204f14f58bdac74c8ee80c8c9-00000000000000000000000000000000
I20260926 16:32:39.226221 17437 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952: Tablet Copy: opened 0 blocks and 1 log segments
I20260926 16:32:39.227324 17991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260926 16:32:39.229404 17991 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9: tablet copy: Starting download of 0 data blocks...
I20260926 16:32:39.229595 17991 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9: tablet copy: Starting download of 1 WAL segments...
I20260926 16:32:39.230934 17991 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260926 16:32:39.234609 17437 tablet_copy_service.cc:342] P d1aa72ae6508448486ff84262f3af952: Request end of tablet copy session ff1d5a9204f14f58bdac74c8ee80c8c9-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.11.8.252@KRBTEST.COM'} at 127.0.0.1:38504
I20260926 16:32:39.234694 17437 tablet_copy_service.cc:433] P d1aa72ae6508448486ff84262f3af952: ending tablet copy session ff1d5a9204f14f58bdac74c8ee80c8c9-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer ff1d5a9204f14f58bdac74c8ee80c8c9
I20260926 16:32:39.242627 17991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:39.244254 18182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:39.244288 18185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:39.244580 18183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:39.244661 17991 server_base.cc:1034] running on GCE node
I20260926 16:32:39.244900 17991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:39.245198 17991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:39.246327 17991 hybrid_clock.cc:648] HybridClock initialized: now 1790440359246320 us; error 24 us; skew 500 ppm
Sep 26 16:32:39 dist-test-slave-2zvq krb5kdc[17372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440359, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:39.247717 17991 init.cc:377] Logged in from keytab as oryx/127.11.8.252@KRBTEST.COM (short username oryx)
I20260926 16:32:39.248164 17991 webserver.cc:460] Webserver started at http://127.11.8.252:34865/ using document root <none> and password file <none>
I20260926 16:32:39.248343 17991 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:39.248394 17991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:39.249420 17991 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 16:32:39.250311 18192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:39.250559 17991 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 16:32:39.250649 17991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-2/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-2/wal
uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9"
format_stamp: "Formatted at 2026-09-26 16:32:39 on dist-test-slave-2zvq"
server_key: "f3206075ea1f557c703521aa1560e763"
server_key_iv: "41783b48a75524e6b280e8153da87e89"
server_key_version: "encryptionkey@0"
I20260926 16:32:39.250727 17991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:39.255139 17742 heartbeater.cc:503] Master 127.11.8.254:32843 was elected leader, sending a full tablet report...
I20260926 16:32:39.265962 17991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:39.268127 17991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:39.272771 17991 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.252:40855
I20260926 16:32:39.273521 17991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/master-2/data/info.pb
I20260926 16:32:39.273908 18244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.252:40855 every 8 connection(s)
I20260926 16:32:39.275224 18245 sys_catalog.cc:263] Verifying existing consensus state
I20260926 16:32:39.276058 18245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9: Bootstrap starting.
I20260926 16:32:39.283172 18245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 16:32:39.283450 18245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9: Bootstrap complete.
I20260926 16:32:39.283908 18245 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: false } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: NON_VOTER last_known_addr { host: "127.11.8.252" port: 40855 } attrs { promote: true } }
I20260926 16:32:39.284036 18245 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ff1d5a9204f14f58bdac74c8ee80c8c9, State: Initialized, Role: LEARNER
I20260926 16:32:39.284157 18245 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: false } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: NON_VOTER last_known_addr { host: "127.11.8.252" port: 40855 } attrs { promote: true } }
I20260926 16:32:39.284451 18248 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: false } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: NON_VOTER last_known_addr { host: "127.11.8.252" port: 40855 } attrs { promote: true } } }
I20260926 16:32:39.284623 18245 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:32:39.284629 18248 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [sys.catalog]: This master's current role is: LEARNER
I20260926 16:32:39.287763 18259 catalog_manager.cc:1259] Loaded cluster ID: 2d3a2295b7af482ea4f34e28e17b0044
I20260926 16:32:39.287868 18259 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9: loading cluster ID for follower catalog manager: success
I20260926 16:32:39.289371 18259 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9: acquiring CA information for follower catalog manager: success
I20260926 16:32:39.290591 18259 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260926 16:32:39.322804 18164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:39.323151 18164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:39.323217 18164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:39.329337 18164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:39.329421 18164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:39.329516 18164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.193
I20260926 16:32:39.336688 18164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:40813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.193.keytab
--principal=oryx/127.11.8.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.193:42459
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.193
--webserver_port=37679
--tserver_master_addrs=127.11.8.254:32843,127.11.8.253:38689,127.11.8.252:40855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.193
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:39.337729 18164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:39.338814 18164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:39.348259 18265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:39.348428 18164 server_base.cc:1034] running on GCE node
W20260926 16:32:39.348321 18264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:39.348259 18267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:39.349005 18164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:39.349596 18164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:39.350769 18164 hybrid_clock.cc:648] HybridClock initialized: now 1790440359350744 us; error 29 us; skew 500 ppm
Sep 26 16:32:39 dist-test-slave-2zvq krb5kdc[17372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440359, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:39.353544 18164 init.cc:377] Logged in from keytab as oryx/127.11.8.193@KRBTEST.COM (short username oryx)
I20260926 16:32:39.354451 18164 webserver.cc:460] Webserver started at http://127.11.8.193:37679/ using document root <none> and password file <none>
I20260926 16:32:39.354978 18164 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:39.355048 18164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:39.358840 18164 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260926 16:32:39.360870 18274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:39.362144 18164 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:32:39.362282 18164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-0/wal
uuid: "452825c51b3246d1acaef2b0bed22348"
format_stamp: "Formatted at 2026-09-26 16:32:37 on dist-test-slave-2zvq"
server_key: "922bb905b6b8fbbcb9dc95413ee1cfba"
server_key_iv: "b15f2d6999cb733941094d9a23956d64"
server_key_version: "encryptionkey@0"
I20260926 16:32:39.362844 18164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:39.396183 18164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:39.399453 18164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:39.399658 18164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:39.400276 18164 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:32:39.401288 18164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:39.401335 18164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:39.401403 18164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:39.401435 18164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:39.410648 18164 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.193:42459
I20260926 16:32:39.410673 18387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.193:42459 every 8 connection(s)
I20260926 16:32:39.411669 18164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
I20260926 16:32:39.414410 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 18164
I20260926 16:32:39.414604 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 17613
I20260926 16:32:39.422606 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.194:45431
--local_ip_for_outbound_sockets=127.11.8.194
--tserver_master_addrs=127.11.8.254:32843
--webserver_port=45933
--webserver_interface=127.11.8.194
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.194.keytab
--principal=oryx/127.11.8.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:40813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.8.254:32843,127.11.8.253:38689
--tserver_master_addrs=127.11.8.254:32843,127.11.8.253:38689,127.11.8.252:40855 with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 16:32:39 dist-test-slave-2zvq krb5kdc[17372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440359, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.193@KRBTEST.COM for oryx/127.11.8.252@KRBTEST.COM
Sep 26 16:32:39 dist-test-slave-2zvq krb5kdc[17372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440359, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.193@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM
Sep 26 16:32:39 dist-test-slave-2zvq krb5kdc[17372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440359, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.193@KRBTEST.COM for oryx/127.11.8.253@KRBTEST.COM
I20260926 16:32:39.430882 17456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.414190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:36895 (local address 127.11.8.254:32843)
0926 16:32:39.414343 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:39.414347 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:39.415942 (+ 1595us) server_negotiation.cc:407] Connection header received
0926 16:32:39.416972 (+ 1030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:39.416976 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:39.417024 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:39.417081 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:39.419105 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.419924 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.422023 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.422395 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.422476 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.426184 (+ 3708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:39.426188 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:39.426191 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:39.426219 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:39.428933 (+ 2714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:39.429445 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:39.429449 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:39.429450 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:39.429531 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:39.429833 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:39.429839 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:39.429841 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:39.430049 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:39.430136 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:39.430589 (+ 453us) server_negotiation.cc:299] Negotiation successful
0926 16:32:39.430711 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 16:32:39.430882 18141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.415559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:42821 (local address 127.11.8.253:38689)
0926 16:32:39.415698 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:39.415704 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:39.415977 (+ 273us) server_negotiation.cc:407] Connection header received
0926 16:32:39.417113 (+ 1136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:39.417120 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:39.417183 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:39.417243 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:39.419101 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.420008 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.421996 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.422354 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.422440 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.426533 (+ 4093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:39.426537 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:39.426541 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:39.426571 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:39.428434 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:39.429049 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:39.429055 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:39.429059 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:39.429148 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:39.429458 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:39.429461 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:39.429463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:39.429708 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:39.429805 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:39.430589 (+ 784us) server_negotiation.cc:299] Negotiation successful
0926 16:32:39.430711 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 16:32:39.431036 18392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.413501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:46737 (local address 127.11.8.252:40855)
0926 16:32:39.413784 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:39.413789 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:39.415285 (+ 1496us) server_negotiation.cc:407] Connection header received
0926 16:32:39.416948 (+ 1663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:39.416952 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:39.417012 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:39.417096 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:39.419105 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.420164 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.422023 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.422395 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.422475 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.426317 (+ 3842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:39.426323 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:39.426330 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:39.426364 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:39.428194 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:39.428872 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:39.428878 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:39.428883 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:39.428999 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:39.429458 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:39.429464 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:39.429467 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:39.429724 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:39.429810 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:39.430612 (+ 802us) server_negotiation.cc:299] Negotiation successful
0926 16:32:39.430767 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":84,"threads_started":1}
I20260926 16:32:39.432593 18394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.413930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.11.8.193:36895)
0926 16:32:39.415846 (+ 1916us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:39.415862 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:39.416708 (+ 846us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:39.417394 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:39.417403 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:39.418047 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:39.418936 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.418955 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.420091 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.420094 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.421882 (+ 1788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.421893 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.422651 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.422653 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.422923 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.423673 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:39.423704 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:39.425985 (+ 2281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:39.429080 (+ 3095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:39.429083 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:39.429084 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:39.429323 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:39.429645 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:39.429647 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:39.429649 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:39.429729 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:39.430220 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:39.430222 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:39.430320 (+ 98us) client_negotiation.cc:769] Sending connection context
0926 16:32:39.431087 (+ 767us) client_negotiation.cc:240] Negotiation successful
0926 16:32:39.431669 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1808,"mutex_wait_us":842,"spinlock_wait_cycles":67200,"thread_start_us":144,"threads_started":1}
I20260926 16:32:39.432624 18395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.415024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.253:38689 (local address 127.11.8.193:42821)
0926 16:32:39.415895 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:39.415908 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:39.416991 (+ 1083us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:39.417405 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:39.417410 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:39.418035 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:39.418942 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.418961 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.420158 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.420165 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.421867 (+ 1702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.421879 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.422648 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.422652 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.423102 (+ 450us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.423853 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:39.423874 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:39.426370 (+ 2496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:39.428597 (+ 2227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:39.428602 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:39.428604 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:39.428932 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:39.429268 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:39.429271 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:39.429273 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:39.429354 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:39.429947 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:39.429950 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:39.430294 (+ 344us) client_negotiation.cc:769] Sending connection context
0926 16:32:39.431049 (+ 755us) client_negotiation.cc:240] Negotiation successful
0926 16:32:39.431480 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":800,"mutex_wait_us":785,"spinlock_wait_cycles":20736,"thread_start_us":95,"threads_started":1}
I20260926 16:32:39.432910 18393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.413930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:40855 (local address 127.11.8.193:46737)
0926 16:32:39.414589 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:39.414640 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:39.416705 (+ 2065us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:39.417394 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:39.417403 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:39.418047 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:39.418936 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.418955 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.420287 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.420290 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.421882 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.421893 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.422648 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.422652 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.422929 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.423673 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:39.423700 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:39.426116 (+ 2416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:39.428339 (+ 2223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:39.428348 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:39.428366 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:39.428742 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:39.429132 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:39.429137 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:39.429141 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:39.429331 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:39.429905 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:39.429914 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:39.430299 (+ 385us) client_negotiation.cc:769] Sending connection context
0926 16:32:39.431195 (+ 896us) client_negotiation.cc:240] Negotiation successful
0926 16:32:39.431584 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"mutex_wait_us":606,"spinlock_wait_cycles":27008,"thread_start_us":153,"threads_started":1}
I20260926 16:32:39.434055 18388 heartbeater.cc:347] Connected to a master server at 127.11.8.252:40855
I20260926 16:32:39.434098 18390 heartbeater.cc:347] Connected to a master server at 127.11.8.253:38689
I20260926 16:32:39.434099 18389 heartbeater.cc:347] Connected to a master server at 127.11.8.254:32843
I20260926 16:32:39.434348 18388 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:39.434348 18389 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:39.434348 18390 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:39.434934 18389 heartbeater.cc:511] Master 127.11.8.254:32843 requested a full tablet report, sending...
I20260926 16:32:39.434935 18388 heartbeater.cc:511] Master 127.11.8.252:40855 requested a full tablet report, sending...
I20260926 16:32:39.434935 18390 heartbeater.cc:511] Master 127.11.8.253:38689 requested a full tablet report, sending...
I20260926 16:32:39.436331 17417 ts_manager.cc:194] Re-registered known tserver with Master: 452825c51b3246d1acaef2b0bed22348 (127.11.8.193:42459)
I20260926 16:32:39.436712 18209 ts_manager.cc:194] Registered new tserver with Master: 452825c51b3246d1acaef2b0bed22348 (127.11.8.193:42459)
I20260926 16:32:39.436769 18019 ts_manager.cc:194] Registered new tserver with Master: 452825c51b3246d1acaef2b0bed22348 (127.11.8.193:42459)
I20260926 16:32:39.437114 17417 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.8.193@KRBTEST.COM'} at 127.11.8.193:36895
I20260926 16:32:39.504932 17876 heartbeater.cc:503] Master 127.11.8.254:32843 was elected leader, sending a full tablet report...
W20260926 16:32:39.540143 18396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:39.540467 18396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:39.540534 18396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:39.544454 18396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:39.544508 18396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:39.544600 18396 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.194
I20260926 16:32:39.549772 18396 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:40813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.194.keytab
--principal=oryx/127.11.8.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.194:45431
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.194
--webserver_port=45933
--tserver_master_addrs=127.11.8.254:32843,127.11.8.253:38689,127.11.8.252:40855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18396
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.194
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:39.550809 18396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:39.551812 18396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:39.561661 18402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:39.561678 18401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:39.561726 18396 server_base.cc:1034] running on GCE node
W20260926 16:32:39.561676 18404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:39.562247 18396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:39.562924 18396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:39.564113 18396 hybrid_clock.cc:648] HybridClock initialized: now 1790440359564073 us; error 52 us; skew 500 ppm
Sep 26 16:32:39 dist-test-slave-2zvq krb5kdc[17372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440359, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:39.566919 18396 init.cc:377] Logged in from keytab as oryx/127.11.8.194@KRBTEST.COM (short username oryx)
I20260926 16:32:39.567830 18396 webserver.cc:460] Webserver started at http://127.11.8.194:45933/ using document root <none> and password file <none>
I20260926 16:32:39.568377 18396 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:39.568476 18396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:39.572731 18396 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 16:32:39.574746 18411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:39.575951 18396 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:32:39.576103 18396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-1/wal
uuid: "beada9654a9f49fa98d63053a48a77da"
format_stamp: "Formatted at 2026-09-26 16:32:38 on dist-test-slave-2zvq"
server_key: "8e5fd1cc4338e86425ce5c0007aee19d"
server_key_iv: "1bdc62d301b2bd35e5101c2cca019f6c"
server_key_version: "encryptionkey@0"
I20260926 16:32:39.576619 18396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:39.591311 18396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:39.594516 18396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:39.594722 18396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:39.595335 18396 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:32:39.596249 18396 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:39.596320 18396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:39.596397 18396 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:39.596439 18396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:39.606076 18396 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.194:45431
I20260926 16:32:39.606096 18524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.194:45431 every 8 connection(s)
I20260926 16:32:39.607213 18396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
I20260926 16:32:39.610576 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 18396
I20260926 16:32:39.610757 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 17747
Sep 26 16:32:39 dist-test-slave-2zvq krb5kdc[17372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440359, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.194@KRBTEST.COM for oryx/127.11.8.252@KRBTEST.COM
Sep 26 16:32:39 dist-test-slave-2zvq krb5kdc[17372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440359, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.194@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM
I20260926 16:32:39.618783 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.195:39875
--local_ip_for_outbound_sockets=127.11.8.195
--tserver_master_addrs=127.11.8.254:32843
--webserver_port=34649
--webserver_interface=127.11.8.195
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.195.keytab
--principal=oryx/127.11.8.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:40813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.8.254:32843,127.11.8.253:38689
--tserver_master_addrs=127.11.8.254:32843,127.11.8.253:38689,127.11.8.252:40855 with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 16:32:39 dist-test-slave-2zvq krb5kdc[17372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440359, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.194@KRBTEST.COM for oryx/127.11.8.253@KRBTEST.COM
I20260926 16:32:39.623988 17456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.609845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:57015 (local address 127.11.8.254:32843)
0926 16:32:39.609960 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:39.609964 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:39.610179 (+ 215us) server_negotiation.cc:407] Connection header received
0926 16:32:39.611520 (+ 1341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:39.611527 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:39.611594 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:39.611654 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:39.613670 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.614618 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.616073 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.616434 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.616561 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.619388 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:39.619395 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:39.619402 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:39.619436 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:39.621290 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:39.621990 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:39.621994 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:39.621996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:39.622079 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:39.622519 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:39.622522 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:39.622524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:39.622750 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:39.622823 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:39.623604 (+ 781us) server_negotiation.cc:299] Negotiation successful
0926 16:32:39.623748 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 16:32:39.624004 18392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.609623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:51995 (local address 127.11.8.252:40855)
0926 16:32:39.609735 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:39.609742 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:39.610180 (+ 438us) server_negotiation.cc:407] Connection header received
0926 16:32:39.611524 (+ 1344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:39.611531 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:39.611619 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:39.611679 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:39.613649 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.614185 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.615344 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.615601 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.615712 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.619382 (+ 3670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:39.619386 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:39.619389 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:39.619416 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:39.621531 (+ 2115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:39.622127 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:39.622132 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:39.622134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:39.622227 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:39.622658 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:39.622664 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:39.622669 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:39.622908 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:39.622981 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:39.623605 (+ 624us) server_negotiation.cc:299] Negotiation successful
0926 16:32:39.623765 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 16:32:39.625684 18530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.609425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.11.8.194:57015)
0926 16:32:39.610026 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:39.610082 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:39.611273 (+ 1191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:39.612075 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:39.612086 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:39.612690 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:39.613509 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.613526 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.614765 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.614771 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.615933 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.615946 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.616685 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.616688 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.616785 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.617304 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:39.617324 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:39.619182 (+ 1858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:39.621418 (+ 2236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:39.621427 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:39.621447 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:39.621845 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:39.622195 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:39.622199 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:39.622204 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:39.622400 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:39.622911 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:39.622922 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:39.623296 (+ 374us) client_negotiation.cc:769] Sending connection context
0926 16:32:39.624161 (+ 865us) client_negotiation.cc:240] Negotiation successful
0926 16:32:39.624530 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"mutex_wait_us":91,"spinlock_wait_cycles":23680,"thread_start_us":140,"threads_started":1}
I20260926 16:32:39.625972 18531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.609425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.253:38689 (local address 127.11.8.194:50939)
0926 16:32:39.616196 (+ 6771us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:39.616212 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:39.616360 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:39.616736 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:39.616741 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:39.617027 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:39.617174 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.617183 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.618375 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.618381 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.619224 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.619230 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.619974 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.619991 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.620120 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.621030 (+ 910us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:39.621048 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:39.622533 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:39.624482 (+ 1949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:39.624489 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:39.624493 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:39.624707 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:39.624997 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:39.625000 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:39.625002 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:39.625081 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:39.625513 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:39.625516 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:39.625596 (+ 80us) client_negotiation.cc:769] Sending connection context
0926 16:32:39.625682 (+ 86us) client_negotiation.cc:240] Negotiation successful
0926 16:32:39.625809 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6667,"mutex_wait_us":2792,"thread_start_us":102,"threads_started":1}
I20260926 16:32:39.626578 18141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.608996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:50939 (local address 127.11.8.253:38689)
0926 16:32:39.609131 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:39.609135 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:39.616287 (+ 7152us) server_negotiation.cc:407] Connection header received
0926 16:32:39.616465 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:39.616471 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:39.616528 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:39.616585 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:39.617317 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.618189 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.619354 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.619704 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.619791 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.622672 (+ 2881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:39.622675 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:39.622678 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:39.622703 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:39.624355 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:39.624813 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:39.624817 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:39.624818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:39.624899 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:39.625173 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:39.625176 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:39.625177 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:39.625374 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:39.625471 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:39.626360 (+ 889us) server_negotiation.cc:299] Negotiation successful
0926 16:32:39.626475 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 16:32:39.627213 18526 heartbeater.cc:347] Connected to a master server at 127.11.8.254:32843
I20260926 16:32:39.627453 18526 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:39.627506 18529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.609425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:40855 (local address 127.11.8.194:51995)
0926 16:32:39.610084 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:39.610095 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:39.611279 (+ 1184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:39.612075 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:39.612087 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:39.612685 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:39.613507 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.613522 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.614354 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.614357 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.615220 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.615229 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.615744 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.615748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.615961 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.616627 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:39.616651 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:39.619212 (+ 2561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:39.621665 (+ 2453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:39.621671 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:39.621673 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:39.622003 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:39.622429 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:39.622435 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:39.622440 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:39.622552 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:39.623056 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:39.623061 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:39.623292 (+ 231us) client_negotiation.cc:769] Sending connection context
0926 16:32:39.624721 (+ 1429us) client_negotiation.cc:240] Negotiation successful
0926 16:32:39.624899 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"mutex_wait_us":225,"spinlock_wait_cycles":52736,"thread_start_us":127,"threads_started":1}
I20260926 16:32:39.628029 18526 heartbeater.cc:511] Master 127.11.8.254:32843 requested a full tablet report, sending...
I20260926 16:32:39.628268 18525 heartbeater.cc:347] Connected to a master server at 127.11.8.252:40855
I20260926 16:32:39.628345 18525 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:39.628484 18525 heartbeater.cc:511] Master 127.11.8.252:40855 requested a full tablet report, sending...
I20260926 16:32:39.628818 18527 heartbeater.cc:347] Connected to a master server at 127.11.8.253:38689
I20260926 16:32:39.628894 18527 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:39.629036 18527 heartbeater.cc:511] Master 127.11.8.253:38689 requested a full tablet report, sending...
I20260926 16:32:39.629158 17417 ts_manager.cc:194] Re-registered known tserver with Master: beada9654a9f49fa98d63053a48a77da (127.11.8.194:45431)
I20260926 16:32:39.629300 18209 ts_manager.cc:194] Registered new tserver with Master: beada9654a9f49fa98d63053a48a77da (127.11.8.194:45431)
I20260926 16:32:39.629683 18019 ts_manager.cc:194] Registered new tserver with Master: beada9654a9f49fa98d63053a48a77da (127.11.8.194:45431)
I20260926 16:32:39.629853 17417 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.8.194@KRBTEST.COM'} at 127.11.8.194:57015
W20260926 16:32:39.737637 18532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:39.737941 18532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:39.738044 18532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:39.742233 18532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:39.742372 18532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:39.742463 18532 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.195
I20260926 16:32:39.747846 18532 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:40813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.195.keytab
--principal=oryx/127.11.8.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.195:39875
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.195
--webserver_port=34649
--tserver_master_addrs=127.11.8.254:32843,127.11.8.253:38689,127.11.8.252:40855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18532
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.195
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:39.748960 18532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:39.750027 18532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:39.761122 18537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:39.761132 18538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:39.761111 18540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:39.761346 18532 server_base.cc:1034] running on GCE node
I20260926 16:32:39.762059 18532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:39.762769 18532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:39.763964 18532 hybrid_clock.cc:648] HybridClock initialized: now 1790440359763941 us; error 31 us; skew 500 ppm
Sep 26 16:32:39 dist-test-slave-2zvq krb5kdc[17372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440359, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:39.766899 18532 init.cc:377] Logged in from keytab as oryx/127.11.8.195@KRBTEST.COM (short username oryx)
I20260926 16:32:39.767935 18532 webserver.cc:460] Webserver started at http://127.11.8.195:34649/ using document root <none> and password file <none>
I20260926 16:32:39.768549 18532 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:39.768646 18532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:39.772516 18532 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 16:32:39.774892 18547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:39.776317 18532 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:32:39.776432 18532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-2/wal
uuid: "fe17e60ac69141458c23a427d88a8ca9"
format_stamp: "Formatted at 2026-09-26 16:32:38 on dist-test-slave-2zvq"
server_key: "2adf451f83c6e0c0f8169a05b2cb6554"
server_key_iv: "5f7a2d901d9f94fa28a9acdf406af69c"
server_key_version: "encryptionkey@0"
I20260926 16:32:39.777127 18532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:39.778738 18392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.774722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53968 (local address 127.11.8.252:40855)
0926 16:32:39.774849 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:39.774853 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:39.775004 (+ 151us) server_negotiation.cc:407] Connection header received
0926 16:32:39.775138 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:39.775147 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:39.775200 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:39.775300 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:39.775565 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.776096 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.777618 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.778167 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.778254 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.778435 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:39.778581 (+ 146us) server_negotiation.cc:299] Negotiation successful
0926 16:32:39.778638 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 16:32:39.778815 18548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.774648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:40855 (local address 127.0.0.1:53968)
0926 16:32:39.774875 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:39.774892 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:39.775028 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:39.775315 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:39.775318 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:39.775338 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:39.775460 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.775465 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.776264 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.776268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.777471 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.777483 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.778327 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.778331 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.778442 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.778479 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 16:32:39.778570 (+ 91us) client_negotiation.cc:240] Negotiation successful
0926 16:32:39.778660 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":85,"threads_started":1}
I20260926 16:32:39.779441 18219 raft_consensus.cc:1203] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260926 16:32:39.779693 18248 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1aa72ae6508448486ff84262f3af952. Latest consensus state: current_term: 1 leader_uuid: "d1aa72ae6508448486ff84262f3af952" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: false } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: NON_VOTER last_known_addr { host: "127.11.8.252" port: 40855 } attrs { promote: true } } }
I20260926 16:32:39.779876 18248 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [sys.catalog]: This master's current role is: LEARNER
I20260926 16:32:39.797833 18532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:39.801834 18532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:39.802043 18532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:39.802731 18532 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:32:39.803632 18532 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:39.803704 18532 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:39.803769 18532 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:39.803798 18532 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:39.813416 18532 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.195:39875
I20260926 16:32:39.813427 18661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.195:39875 every 8 connection(s)
I20260926 16:32:39.814626 18532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
I20260926 16:32:39.817056 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 18532
Sep 26 16:32:39 dist-test-slave-2zvq krb5kdc[17372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440359, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.195@KRBTEST.COM for oryx/127.11.8.253@KRBTEST.COM
Sep 26 16:32:39 dist-test-slave-2zvq krb5kdc[17372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440359, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.195@KRBTEST.COM for oryx/127.11.8.252@KRBTEST.COM
Sep 26 16:32:39 dist-test-slave-2zvq krb5kdc[17372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440359, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.195@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM
I20260926 16:32:39.833635 18674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.819128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53972 (local address 127.11.8.252:40855)
0926 16:32:39.820955 (+ 1827us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:39.820959 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:39.820971 (+ 12us) server_negotiation.cc:407] Connection header received
0926 16:32:39.821022 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:39.821024 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:39.821074 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:39.823991 (+ 2917us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:39.824105 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.824834 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.827013 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.827217 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.827322 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.829214 (+ 1892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:39.829217 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:39.829220 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:39.829251 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:39.831659 (+ 2408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:39.832042 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:39.832049 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:39.832054 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:39.832138 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:39.832468 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:39.832471 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:39.832473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:39.832717 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:39.832838 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:39.833243 (+ 405us) server_negotiation.cc:299] Negotiation successful
0926 16:32:39.833423 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1762,"thread_start_us":89,"threads_started":1}
I20260926 16:32:39.835804 18141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.816497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:34297 (local address 127.11.8.253:38689)
0926 16:32:39.816649 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:39.816655 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:39.819707 (+ 3052us) server_negotiation.cc:407] Connection header received
0926 16:32:39.819798 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:39.819804 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:39.819856 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:39.819930 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:39.821536 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.822371 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.824398 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.824733 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.824805 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.829462 (+ 4657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:39.829468 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:39.829471 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:39.829505 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:39.831509 (+ 2004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:39.832448 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:39.832462 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:39.832463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:39.832595 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:39.834400 (+ 1805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:39.834405 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:39.834406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:39.834656 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:39.834736 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:39.835498 (+ 762us) server_negotiation.cc:299] Negotiation successful
0926 16:32:39.835629 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 16:32:39.836216 18673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.819037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38510 (local address 127.11.8.254:32843)
0926 16:32:39.820583 (+ 1546us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:39.820588 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:39.820603 (+ 15us) server_negotiation.cc:407] Connection header received
0926 16:32:39.820640 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:39.820644 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:39.820686 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:39.820773 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:39.823030 (+ 2257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.823845 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.827423 (+ 3578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.827772 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.827919 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.831234 (+ 3315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:39.831239 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:39.831243 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:39.831272 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:39.833401 (+ 2129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:39.834516 (+ 1115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:39.834520 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:39.834522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:39.834609 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:39.834921 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:39.834924 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:39.834926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:39.835148 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:39.835275 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:39.835913 (+ 638us) server_negotiation.cc:299] Negotiation successful
0926 16:32:39.836066 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1468,"thread_start_us":60,"threads_started":1}
I20260926 16:32:39.837491 18668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.816833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.253:38689 (local address 127.11.8.195:34297)
0926 16:32:39.819599 (+ 2766us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:39.819612 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:39.819718 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:39.820038 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:39.820041 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:39.820590 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:39.821369 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.821384 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.822542 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.822545 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.824262 (+ 1717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.824272 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.824987 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.824991 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.825359 (+ 368us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.826026 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:39.826044 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:39.829093 (+ 3049us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:39.831639 (+ 2546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:39.831642 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:39.831645 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:39.832269 (+ 624us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:39.832768 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:39.832774 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:39.832777 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:39.834142 (+ 1365us) spinlock_profiling.cc:243] Waited 922 us on lock 0x1c6826b8. stack: 00007fe312e46c78 00007fe312e471a9 00007fe312a091d7 00000000005984a6 00000000005984f9 000000000059c488 00007fe312e432ab 00007fe312e429da 00007fe31751b0c4 00007fe31749ecbe 00007fe31749e2c8 00007fe31749d146 00007fe317499e17 00007fe3174da832 00007fe3174db68c 00007fe3174f10d3
0926 16:32:39.834243 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:39.834823 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:39.834831 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:39.835197 (+ 366us) client_negotiation.cc:769] Sending connection context
0926 16:32:39.836008 (+ 811us) client_negotiation.cc:240] Negotiation successful
0926 16:32:39.836389 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2687,"mutex_wait_us":482,"spinlock_wait_cycles":2143744,"thread_start_us":141,"threads_started":1}
I20260926 16:32:39.838303 17456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.816420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:59427 (local address 127.11.8.254:32843)
0926 16:32:39.816575 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:39.816579 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:39.818239 (+ 1660us) server_negotiation.cc:407] Connection header received
0926 16:32:39.819413 (+ 1174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:39.819418 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:39.819492 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:39.819579 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:39.821535 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.822367 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.824398 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.824733 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.824804 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.833759 (+ 8955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:39.833765 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:39.833769 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:39.833804 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:39.835573 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:39.836425 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:39.836429 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:39.836430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:39.836507 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:39.837331 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:39.837335 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:39.837337 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:39.837553 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:39.837645 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:39.837972 (+ 327us) server_negotiation.cc:299] Negotiation successful
0926 16:32:39.838115 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 16:32:39.838766 18664 heartbeater.cc:347] Connected to a master server at 127.11.8.253:38689
I20260926 16:32:39.838835 18667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.816832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.11.8.195:59427)
0926 16:32:39.818087 (+ 1255us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:39.818147 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:39.819191 (+ 1044us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:39.820013 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:39.820022 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:39.820587 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:39.821368 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.821384 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.822542 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.822545 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.824271 (+ 1726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.824278 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.824987 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.824991 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.825367 (+ 376us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.826350 (+ 983us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:39.826365 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:39.833616 (+ 7251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:39.836073 (+ 2457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:39.836076 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:39.836079 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:39.836336 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:39.837126 (+ 790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:39.837132 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:39.837137 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:39.837230 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:39.837690 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:39.837694 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:39.837784 (+ 90us) client_negotiation.cc:769] Sending connection context
0926 16:32:39.838478 (+ 694us) client_negotiation.cc:240] Negotiation successful
0926 16:32:39.838625 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1026,"mutex_wait_us":569,"spinlock_wait_cycles":17152,"thread_start_us":129,"threads_started":1}
I20260926 16:32:39.839048 18664 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:39.839455 18666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.817111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:40855 (local address 127.11.8.195:48739)
0926 16:32:39.819955 (+ 2844us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:39.819969 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:39.820064 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:39.820412 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:39.820416 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:39.820661 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:39.821363 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.821386 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.822538 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.822544 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.824262 (+ 1718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.824271 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.824987 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.824991 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.825294 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.825887 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:39.825912 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:39.829295 (+ 3383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:39.831601 (+ 2306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:39.831607 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:39.831626 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:39.832026 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:39.832393 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:39.832396 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:39.832398 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:39.833610 (+ 1212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:39.838874 (+ 5264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:39.838877 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:39.838971 (+ 94us) client_negotiation.cc:769] Sending connection context
0926 16:32:39.839142 (+ 171us) client_negotiation.cc:240] Negotiation successful
0926 16:32:39.839287 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2764,"mutex_wait_us":510,"spinlock_wait_cycles":7296,"thread_start_us":110,"threads_started":1}
I20260926 16:32:39.839470 18392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.817844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:48739 (local address 127.11.8.252:40855)
0926 16:32:39.817975 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:39.817981 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:39.820055 (+ 2074us) server_negotiation.cc:407] Connection header received
0926 16:32:39.820145 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:39.820151 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:39.820207 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:39.820282 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:39.821552 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.822345 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.824386 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.824732 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.824798 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.829423 (+ 4625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:39.829428 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:39.829431 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:39.829457 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:39.831445 (+ 1988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:39.832165 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:39.832170 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:39.832173 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:39.832263 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:39.835691 (+ 3428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:39.835695 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:39.835697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:39.835920 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:39.836031 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:39.839206 (+ 3175us) server_negotiation.cc:299] Negotiation successful
0926 16:32:39.839327 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":56}
I20260926 16:32:39.839605 18664 heartbeater.cc:511] Master 127.11.8.253:38689 requested a full tablet report, sending...
I20260926 16:32:39.839972 18662 heartbeater.cc:347] Connected to a master server at 127.11.8.252:40855
I20260926 16:32:39.840060 18662 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:39.840233 18662 heartbeater.cc:511] Master 127.11.8.252:40855 requested a full tablet report, sending...
I20260926 16:32:39.840553 18663 heartbeater.cc:347] Connected to a master server at 127.11.8.254:32843
I20260926 16:32:39.840620 18663 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:39.840758 18663 heartbeater.cc:511] Master 127.11.8.254:32843 requested a full tablet report, sending...
I20260926 16:32:39.840756 18019 ts_manager.cc:194] Registered new tserver with Master: fe17e60ac69141458c23a427d88a8ca9 (127.11.8.195:39875)
I20260926 16:32:39.841262 17416 ts_manager.cc:194] Re-registered known tserver with Master: fe17e60ac69141458c23a427d88a8ca9 (127.11.8.195:39875)
I20260926 16:32:39.841357 18209 ts_manager.cc:194] Registered new tserver with Master: fe17e60ac69141458c23a427d88a8ca9 (127.11.8.195:39875)
I20260926 16:32:39.841930 17416 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.8.195@KRBTEST.COM'} at 127.11.8.195:59427
I20260926 16:32:39.839309 17417 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38510:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:32:39.843292 17417 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:32:39.845975 18681 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d1aa72ae6508448486ff84262f3af952" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: false } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: NON_VOTER last_known_addr { host: "127.11.8.252" port: 40855 } attrs { promote: true } } }
I20260926 16:32:39.846110 18681 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:39.848289 18675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.819171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36502 (local address 127.11.8.253:38689)
0926 16:32:39.822144 (+ 2973us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:39.822148 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:39.828270 (+ 6122us) server_negotiation.cc:407] Connection header received
0926 16:32:39.828479 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:39.828482 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:39.828534 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:39.828662 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:39.834889 (+ 6227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.835758 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.842883 (+ 7125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.843159 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.843269 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.844714 (+ 1445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:39.844718 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:39.844721 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:39.844742 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:39.846520 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:39.846992 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:39.846994 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:39.846996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:39.847079 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:39.847399 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:39.847402 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:39.847403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:39.847665 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:39.847792 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:39.848002 (+ 210us) server_negotiation.cc:299] Negotiation successful
0926 16:32:39.848139 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2901,"thread_start_us":91,"threads_started":1}
I20260926 16:32:39.849079 18681 raft_consensus.cc:1050] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952: attempting to promote NON_VOTER ff1d5a9204f14f58bdac74c8ee80c8c9 to VOTER
I20260926 16:32:39.849484 18681 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: false } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 40855 } attrs { promote: false } }
I20260926 16:32:39.850747 18029 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [term 1 FOLLOWER]: Refusing update from remote peer d1aa72ae6508448486ff84262f3af952: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260926 16:32:39.850759 18219 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [term 1 LEARNER]: Refusing update from remote peer d1aa72ae6508448486ff84262f3af952: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260926 16:32:39.851133 18687 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260926 16:32:39.851300 18680 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 40855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260926 16:32:39.853155 18681 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, ff1d5a9204f14f58bdac74c8ee80c8c9 (127.11.8.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: false } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 40855 } attrs { promote: false } } }
I20260926 16:32:39.853379 18029 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, ff1d5a9204f14f58bdac74c8ee80c8c9 (127.11.8.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: false } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 40855 } attrs { promote: false } } }
I20260926 16:32:39.854686 17978 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d1aa72ae6508448486ff84262f3af952" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: false } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 40855 } attrs { promote: false } } }
I20260926 16:32:39.854816 17978 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:39.855198 18681 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d1aa72ae6508448486ff84262f3af952" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: false } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 40855 } attrs { promote: false } } }
I20260926 16:32:39.855304 18681 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:39.855484 18681 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d1aa72ae6508448486ff84262f3af952" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: false } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 40855 } attrs { promote: false } } }
I20260926 16:32:39.855575 18681 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952 [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:39.855895 18690 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d1aa72ae6508448486ff84262f3af952" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: false } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 40855 } attrs { promote: false } } }
I20260926 16:32:39.856031 18690 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:32:39.856261 18219 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, ff1d5a9204f14f58bdac74c8ee80c8c9 (127.11.8.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: false } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 40855 } attrs { promote: false } } }
I20260926 16:32:39.857803 18248 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d1aa72ae6508448486ff84262f3af952" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: false } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 40855 } attrs { promote: false } } }
I20260926 16:32:39.857914 18248 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:32:39.864661 18694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.858474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:42459 (local address 127.0.0.1:33416)
0926 16:32:39.859042 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:39.859057 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:39.859157 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:39.859954 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:39.859957 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:39.859976 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:39.860086 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.860096 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.861996 (+ 1900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.862001 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.863199 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.863209 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.864259 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.864263 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.864363 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.864393 (+ 30us) client_negotiation.cc:769] Sending connection context
0926 16:32:39.864445 (+ 52us) client_negotiation.cc:240] Negotiation successful
0926 16:32:39.864514 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"spinlock_wait_cycles":2816}
I20260926 16:32:39.864848 18548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.858767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:45431 (local address 127.0.0.1:53474)
0926 16:32:39.859044 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:39.859059 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:39.859147 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:39.860096 (+ 949us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:39.860099 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:39.860116 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:39.860208 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.860214 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.862306 (+ 2092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.862309 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.863375 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.863383 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.864422 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.864427 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.864533 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.864581 (+ 48us) client_negotiation.cc:769] Sending connection context
0926 16:32:39.864631 (+ 50us) client_negotiation.cc:240] Negotiation successful
0926 16:32:39.864695 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"spinlock_wait_cycles":896,"thread_start_us":132,"threads_started":1}
I20260926 16:32:39.864987 18696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.859879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:39875 (local address 127.0.0.1:52838)
0926 16:32:39.860983 (+ 1104us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:39.860996 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:39.861097 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:39.861586 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:39.861589 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:39.861607 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:39.861694 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.861699 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.862792 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.862798 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.863609 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.863619 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.864607 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.864613 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.864704 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.864724 (+ 20us) client_negotiation.cc:769] Sending connection context
0926 16:32:39.864800 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 16:32:39.864860 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1051,"thread_start_us":80,"threads_started":1}
I20260926 16:32:39.865090 18697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.860260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52838 (local address 127.11.8.195:39875)
0926 16:32:39.860858 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:39.860868 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:39.861195 (+ 327us) server_negotiation.cc:407] Connection header received
0926 16:32:39.861270 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:39.861276 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:39.861443 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:39.861600 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:39.861783 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.862677 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.863733 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.864497 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.864595 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.864737 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:39.864833 (+ 96us) server_negotiation.cc:299] Negotiation successful
0926 16:32:39.864938 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":104,"threads_started":1}
I20260926 16:32:39.865485 18695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.859259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53474 (local address 127.11.8.194:45431)
0926 16:32:39.859613 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:39.859624 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:39.859647 (+ 23us) server_negotiation.cc:407] Connection header received
0926 16:32:39.859723 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:39.859728 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:39.859926 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:39.860494 (+ 568us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:39.860638 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.862161 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.863502 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.864294 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.865111 (+ 817us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.865261 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:39.865302 (+ 41us) server_negotiation.cc:299] Negotiation successful
0926 16:32:39.865364 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":125,"threads_started":1}
I20260926 16:32:39.865863 18693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.858775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33416 (local address 127.11.8.193:42459)
0926 16:32:39.859422 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:39.859435 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:39.859458 (+ 23us) server_negotiation.cc:407] Connection header received
0926 16:32:39.859538 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:39.859545 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:39.859778 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:39.860249 (+ 471us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:39.860399 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.861891 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.863348 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.864121 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.865461 (+ 1340us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.865589 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:39.865635 (+ 46us) server_negotiation.cc:299] Negotiation successful
0926 16:32:39.865716 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":106,"threads_started":1}
I20260926 16:32:39.868126 18459 tablet_service.cc:1467] Processing CreateTablet for tablet 8b32453a98564720b1f4156961060385 (DEFAULT_TABLE table=test-table [id=0f96d0f5a68b43d2b5edb67047c1d1d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:39.868124 18596 tablet_service.cc:1467] Processing CreateTablet for tablet 8b32453a98564720b1f4156961060385 (DEFAULT_TABLE table=test-table [id=0f96d0f5a68b43d2b5edb67047c1d1d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:39.868459 18322 tablet_service.cc:1467] Processing CreateTablet for tablet 8b32453a98564720b1f4156961060385 (DEFAULT_TABLE table=test-table [id=0f96d0f5a68b43d2b5edb67047c1d1d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:39.869218 18459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b32453a98564720b1f4156961060385. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:39.869218 18596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b32453a98564720b1f4156961060385. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:39.869426 18322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b32453a98564720b1f4156961060385. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:39.876356 18699 tablet_bootstrap.cc:492] T 8b32453a98564720b1f4156961060385 P 452825c51b3246d1acaef2b0bed22348: Bootstrap starting.
I20260926 16:32:39.877660 18700 tablet_bootstrap.cc:492] T 8b32453a98564720b1f4156961060385 P fe17e60ac69141458c23a427d88a8ca9: Bootstrap starting.
I20260926 16:32:39.879107 18699 tablet_bootstrap.cc:654] T 8b32453a98564720b1f4156961060385 P 452825c51b3246d1acaef2b0bed22348: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:39.879776 18700 tablet_bootstrap.cc:654] T 8b32453a98564720b1f4156961060385 P fe17e60ac69141458c23a427d88a8ca9: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:39.880064 18699 log.cc:824] T 8b32453a98564720b1f4156961060385 P 452825c51b3246d1acaef2b0bed22348: Log is configured to *not* fsync() on all Append() calls
I20260926 16:32:39.880486 18700 log.cc:824] T 8b32453a98564720b1f4156961060385 P fe17e60ac69141458c23a427d88a8ca9: Log is configured to *not* fsync() on all Append() calls
I20260926 16:32:39.880965 18701 tablet_bootstrap.cc:492] T 8b32453a98564720b1f4156961060385 P beada9654a9f49fa98d63053a48a77da: Bootstrap starting.
I20260926 16:32:39.882150 18699 tablet_bootstrap.cc:492] T 8b32453a98564720b1f4156961060385 P 452825c51b3246d1acaef2b0bed22348: No bootstrap required, opened a new log
I20260926 16:32:39.882419 18699 ts_tablet_manager.cc:1397] T 8b32453a98564720b1f4156961060385 P 452825c51b3246d1acaef2b0bed22348: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260926 16:32:39.882956 18701 tablet_bootstrap.cc:654] T 8b32453a98564720b1f4156961060385 P beada9654a9f49fa98d63053a48a77da: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:39.883580 18701 log.cc:824] T 8b32453a98564720b1f4156961060385 P beada9654a9f49fa98d63053a48a77da: Log is configured to *not* fsync() on all Append() calls
I20260926 16:32:39.885640 18701 tablet_bootstrap.cc:492] T 8b32453a98564720b1f4156961060385 P beada9654a9f49fa98d63053a48a77da: No bootstrap required, opened a new log
I20260926 16:32:39.885828 18701 ts_tablet_manager.cc:1397] T 8b32453a98564720b1f4156961060385 P beada9654a9f49fa98d63053a48a77da: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260926 16:32:39.886369 18699 raft_consensus.cc:348] T 8b32453a98564720b1f4156961060385 P 452825c51b3246d1acaef2b0bed22348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452825c51b3246d1acaef2b0bed22348" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42459 } } peers { permanent_uuid: "beada9654a9f49fa98d63053a48a77da" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45431 } } peers { permanent_uuid: "fe17e60ac69141458c23a427d88a8ca9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 39875 } }
I20260926 16:32:39.886658 18699 raft_consensus.cc:374] T 8b32453a98564720b1f4156961060385 P 452825c51b3246d1acaef2b0bed22348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:39.886740 18699 raft_consensus.cc:729] T 8b32453a98564720b1f4156961060385 P 452825c51b3246d1acaef2b0bed22348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452825c51b3246d1acaef2b0bed22348, State: Initialized, Role: FOLLOWER
I20260926 16:32:39.887138 18699 consensus_queue.cc:260] T 8b32453a98564720b1f4156961060385 P 452825c51b3246d1acaef2b0bed22348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452825c51b3246d1acaef2b0bed22348" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42459 } } peers { permanent_uuid: "beada9654a9f49fa98d63053a48a77da" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45431 } } peers { permanent_uuid: "fe17e60ac69141458c23a427d88a8ca9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 39875 } }
I20260926 16:32:39.887848 18389 heartbeater.cc:503] Master 127.11.8.254:32843 was elected leader, sending a full tablet report...
I20260926 16:32:39.891448 18701 raft_consensus.cc:348] T 8b32453a98564720b1f4156961060385 P beada9654a9f49fa98d63053a48a77da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452825c51b3246d1acaef2b0bed22348" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42459 } } peers { permanent_uuid: "beada9654a9f49fa98d63053a48a77da" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45431 } } peers { permanent_uuid: "fe17e60ac69141458c23a427d88a8ca9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 39875 } }
I20260926 16:32:39.891671 18701 raft_consensus.cc:374] T 8b32453a98564720b1f4156961060385 P beada9654a9f49fa98d63053a48a77da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:39.891727 18701 raft_consensus.cc:729] T 8b32453a98564720b1f4156961060385 P beada9654a9f49fa98d63053a48a77da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: beada9654a9f49fa98d63053a48a77da, State: Initialized, Role: FOLLOWER
I20260926 16:32:39.892156 18701 consensus_queue.cc:260] T 8b32453a98564720b1f4156961060385 P beada9654a9f49fa98d63053a48a77da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452825c51b3246d1acaef2b0bed22348" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42459 } } peers { permanent_uuid: "beada9654a9f49fa98d63053a48a77da" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45431 } } peers { permanent_uuid: "fe17e60ac69141458c23a427d88a8ca9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 39875 } }
I20260926 16:32:39.893098 18701 ts_tablet_manager.cc:1428] T 8b32453a98564720b1f4156961060385 P beada9654a9f49fa98d63053a48a77da: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260926 16:32:39.888074 18699 ts_tablet_manager.cc:1428] T 8b32453a98564720b1f4156961060385 P 452825c51b3246d1acaef2b0bed22348: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260926 16:32:39.893786 18526 heartbeater.cc:503] Master 127.11.8.254:32843 was elected leader, sending a full tablet report...
I20260926 16:32:39.894784 18700 tablet_bootstrap.cc:492] T 8b32453a98564720b1f4156961060385 P fe17e60ac69141458c23a427d88a8ca9: No bootstrap required, opened a new log
I20260926 16:32:39.894982 18700 ts_tablet_manager.cc:1397] T 8b32453a98564720b1f4156961060385 P fe17e60ac69141458c23a427d88a8ca9: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.000s
I20260926 16:32:39.899209 18700 raft_consensus.cc:348] T 8b32453a98564720b1f4156961060385 P fe17e60ac69141458c23a427d88a8ca9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452825c51b3246d1acaef2b0bed22348" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42459 } } peers { permanent_uuid: "beada9654a9f49fa98d63053a48a77da" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45431 } } peers { permanent_uuid: "fe17e60ac69141458c23a427d88a8ca9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 39875 } }
I20260926 16:32:39.899403 18700 raft_consensus.cc:374] T 8b32453a98564720b1f4156961060385 P fe17e60ac69141458c23a427d88a8ca9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:39.899434 18700 raft_consensus.cc:729] T 8b32453a98564720b1f4156961060385 P fe17e60ac69141458c23a427d88a8ca9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe17e60ac69141458c23a427d88a8ca9, State: Initialized, Role: FOLLOWER
I20260926 16:32:39.899883 18700 consensus_queue.cc:260] T 8b32453a98564720b1f4156961060385 P fe17e60ac69141458c23a427d88a8ca9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452825c51b3246d1acaef2b0bed22348" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42459 } } peers { permanent_uuid: "beada9654a9f49fa98d63053a48a77da" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45431 } } peers { permanent_uuid: "fe17e60ac69141458c23a427d88a8ca9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 39875 } }
I20260926 16:32:39.900738 18700 ts_tablet_manager.cc:1428] T 8b32453a98564720b1f4156961060385 P fe17e60ac69141458c23a427d88a8ca9: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260926 16:32:39.901292 18663 heartbeater.cc:503] Master 127.11.8.254:32843 was elected leader, sending a full tablet report...
W20260926 16:32:39.913790 18391 tablet.cc:2369] T 8b32453a98564720b1f4156961060385 P 452825c51b3246d1acaef2b0bed22348: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:32:39.964555 18706 raft_consensus.cc:482] T 8b32453a98564720b1f4156961060385 P beada9654a9f49fa98d63053a48a77da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:32:39.964815 18706 raft_consensus.cc:504] T 8b32453a98564720b1f4156961060385 P beada9654a9f49fa98d63053a48a77da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452825c51b3246d1acaef2b0bed22348" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42459 } } peers { permanent_uuid: "beada9654a9f49fa98d63053a48a77da" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45431 } } peers { permanent_uuid: "fe17e60ac69141458c23a427d88a8ca9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 39875 } }
I20260926 16:32:39.966074 18706 leader_election.cc:288] T 8b32453a98564720b1f4156961060385 P beada9654a9f49fa98d63053a48a77da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 452825c51b3246d1acaef2b0bed22348 (127.11.8.193:42459), fe17e60ac69141458c23a427d88a8ca9 (127.11.8.195:39875)
I20260926 16:32:39.969982 18697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.966453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:38527 (local address 127.11.8.195:39875)
0926 16:32:39.966601 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:39.966608 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:39.966628 (+ 20us) server_negotiation.cc:407] Connection header received
0926 16:32:39.966694 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:39.966700 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:39.966754 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:39.966883 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:39.967214 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.967819 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.969024 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.969437 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.969567 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.969621 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:39.969786 (+ 165us) server_negotiation.cc:299] Negotiation successful
0926 16:32:39.969853 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 16:32:39.970001 18529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.966341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:39875 (local address 127.11.8.194:38527)
0926 16:32:39.966477 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:39.966493 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:39.966599 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:39.966901 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:39.966904 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:39.966921 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:39.967037 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.967042 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.967932 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.967937 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.968899 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.968908 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.969559 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.969563 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.969656 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.969683 (+ 27us) client_negotiation.cc:769] Sending connection context
0926 16:32:39.969771 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 16:32:39.969853 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260926 16:32:39.970028 18530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.966341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:42459 (local address 127.11.8.194:37563)
0926 16:32:39.966560 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:39.966574 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:39.966661 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:39.967122 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:39.967125 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:39.967140 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:39.967262 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.967289 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.968048 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.968064 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.968954 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:39.968963 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.969614 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.969617 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:39.969715 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.969742 (+ 27us) client_negotiation.cc:769] Sending connection context
0926 16:32:39.969831 (+ 89us) client_negotiation.cc:240] Negotiation successful
0926 16:32:39.969896 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":53}
I20260926 16:32:39.970028 18693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:39.966743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:37563 (local address 127.11.8.193:42459)
0926 16:32:39.966882 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:39.966898 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:39.966911 (+ 13us) server_negotiation.cc:407] Connection header received
0926 16:32:39.966977 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:39.966980 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:39.967032 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:39.967110 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:39.967457 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.967941 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.969082 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:39.969491 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:39.969613 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:39.969662 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:39.969845 (+ 183us) server_negotiation.cc:299] Negotiation successful
0926 16:32:39.969919 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 16:32:39.970662 18616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b32453a98564720b1f4156961060385" candidate_uuid: "beada9654a9f49fa98d63053a48a77da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe17e60ac69141458c23a427d88a8ca9" is_pre_election: true
I20260926 16:32:39.970779 18342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b32453a98564720b1f4156961060385" candidate_uuid: "beada9654a9f49fa98d63053a48a77da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452825c51b3246d1acaef2b0bed22348" is_pre_election: true
I20260926 16:32:39.970976 18616 raft_consensus.cc:2454] T 8b32453a98564720b1f4156961060385 P fe17e60ac69141458c23a427d88a8ca9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate beada9654a9f49fa98d63053a48a77da in term 0.
I20260926 16:32:39.971091 18342 raft_consensus.cc:2454] T 8b32453a98564720b1f4156961060385 P 452825c51b3246d1acaef2b0bed22348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate beada9654a9f49fa98d63053a48a77da in term 0.
I20260926 16:32:39.971472 18414 leader_election.cc:302] T 8b32453a98564720b1f4156961060385 P beada9654a9f49fa98d63053a48a77da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: beada9654a9f49fa98d63053a48a77da, fe17e60ac69141458c23a427d88a8ca9; no voters:
I20260926 16:32:39.971755 18706 raft_consensus.cc:2781] T 8b32453a98564720b1f4156961060385 P beada9654a9f49fa98d63053a48a77da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:32:39.971845 18706 raft_consensus.cc:482] T 8b32453a98564720b1f4156961060385 P beada9654a9f49fa98d63053a48a77da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:32:39.971890 18706 raft_consensus.cc:3037] T 8b32453a98564720b1f4156961060385 P beada9654a9f49fa98d63053a48a77da [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:39.972981 18706 raft_consensus.cc:504] T 8b32453a98564720b1f4156961060385 P beada9654a9f49fa98d63053a48a77da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452825c51b3246d1acaef2b0bed22348" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42459 } } peers { permanent_uuid: "beada9654a9f49fa98d63053a48a77da" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45431 } } peers { permanent_uuid: "fe17e60ac69141458c23a427d88a8ca9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 39875 } }
I20260926 16:32:39.973307 18706 leader_election.cc:288] T 8b32453a98564720b1f4156961060385 P beada9654a9f49fa98d63053a48a77da [CANDIDATE]: Term 1 election: Requested vote from peers 452825c51b3246d1acaef2b0bed22348 (127.11.8.193:42459), fe17e60ac69141458c23a427d88a8ca9 (127.11.8.195:39875)
I20260926 16:32:39.973625 18342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b32453a98564720b1f4156961060385" candidate_uuid: "beada9654a9f49fa98d63053a48a77da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452825c51b3246d1acaef2b0bed22348"
I20260926 16:32:39.973654 18616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b32453a98564720b1f4156961060385" candidate_uuid: "beada9654a9f49fa98d63053a48a77da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe17e60ac69141458c23a427d88a8ca9"
I20260926 16:32:39.973766 18342 raft_consensus.cc:3037] T 8b32453a98564720b1f4156961060385 P 452825c51b3246d1acaef2b0bed22348 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:39.973784 18616 raft_consensus.cc:3037] T 8b32453a98564720b1f4156961060385 P fe17e60ac69141458c23a427d88a8ca9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:39.974804 18342 raft_consensus.cc:2454] T 8b32453a98564720b1f4156961060385 P 452825c51b3246d1acaef2b0bed22348 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate beada9654a9f49fa98d63053a48a77da in term 1.
I20260926 16:32:39.974932 18616 raft_consensus.cc:2454] T 8b32453a98564720b1f4156961060385 P fe17e60ac69141458c23a427d88a8ca9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate beada9654a9f49fa98d63053a48a77da in term 1.
I20260926 16:32:39.975229 18414 leader_election.cc:302] T 8b32453a98564720b1f4156961060385 P beada9654a9f49fa98d63053a48a77da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: beada9654a9f49fa98d63053a48a77da, fe17e60ac69141458c23a427d88a8ca9; no voters:
I20260926 16:32:39.975400 18706 raft_consensus.cc:2781] T 8b32453a98564720b1f4156961060385 P beada9654a9f49fa98d63053a48a77da [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:32:39.975603 18706 raft_consensus.cc:686] T 8b32453a98564720b1f4156961060385 P beada9654a9f49fa98d63053a48a77da [term 1 LEADER]: Becoming Leader. State: Replica: beada9654a9f49fa98d63053a48a77da, State: Running, Role: LEADER
I20260926 16:32:39.975921 18706 consensus_queue.cc:237] T 8b32453a98564720b1f4156961060385 P beada9654a9f49fa98d63053a48a77da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452825c51b3246d1acaef2b0bed22348" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42459 } } peers { permanent_uuid: "beada9654a9f49fa98d63053a48a77da" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45431 } } peers { permanent_uuid: "fe17e60ac69141458c23a427d88a8ca9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 39875 } }
I20260926 16:32:39.979408 17417 catalog_manager.cc:5495] T 8b32453a98564720b1f4156961060385 P beada9654a9f49fa98d63053a48a77da reported cstate change: term changed from 0 to 1, leader changed from <none> to beada9654a9f49fa98d63053a48a77da (127.11.8.194). New cstate: current_term: 1 leader_uuid: "beada9654a9f49fa98d63053a48a77da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "452825c51b3246d1acaef2b0bed22348" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "beada9654a9f49fa98d63053a48a77da" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fe17e60ac69141458c23a427d88a8ca9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 39875 } health_report { overall_health: UNKNOWN } } }
I20260926 16:32:40.014797 18695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:40.011510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53484 (local address 127.11.8.194:45431)
0926 16:32:40.011642 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:40.011645 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:40.011657 (+ 12us) server_negotiation.cc:407] Connection header received
0926 16:32:40.011731 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:40.011734 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:40.011787 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:40.011879 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:32:40.012099 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:40.012595 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:40.013403 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:40.013607 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:40.013685 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:40.013930 (+ 245us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:32:40.014422 (+ 492us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:32:40.014516 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:40.014582 (+ 66us) server_negotiation.cc:299] Negotiation successful
0926 16:32:40.014629 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 16:32:40.021009 18616 raft_consensus.cc:1261] T 8b32453a98564720b1f4156961060385 P fe17e60ac69141458c23a427d88a8ca9 [term 1 FOLLOWER]: Refusing update from remote peer beada9654a9f49fa98d63053a48a77da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:32:40.021013 18342 raft_consensus.cc:1261] T 8b32453a98564720b1f4156961060385 P 452825c51b3246d1acaef2b0bed22348 [term 1 FOLLOWER]: Refusing update from remote peer beada9654a9f49fa98d63053a48a77da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:32:40.021708 18708 consensus_queue.cc:1035] T 8b32453a98564720b1f4156961060385 P beada9654a9f49fa98d63053a48a77da [LEADER]: Connected to new peer: Peer: permanent_uuid: "452825c51b3246d1acaef2b0bed22348" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:32:40.021904 18706 consensus_queue.cc:1035] T 8b32453a98564720b1f4156961060385 P beada9654a9f49fa98d63053a48a77da [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe17e60ac69141458c23a427d88a8ca9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 39875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:32:40.040748 17417 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:32:40.040894 17417 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:32:40.044886 17417 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d1aa72ae6508448486ff84262f3af952: Sending DeleteTablet for 3 replicas of tablet 8b32453a98564720b1f4156961060385
I20260926 16:32:40.045615 18459 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b32453a98564720b1f4156961060385 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:32:40 UTC) from {username='oryx', principal='oryx/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:53474
I20260926 16:32:40.045675 18322 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b32453a98564720b1f4156961060385 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:32:40 UTC) from {username='oryx', principal='oryx/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:33416
I20260926 16:32:40.046129 18722 tablet_replica.cc:330] T 8b32453a98564720b1f4156961060385 P beada9654a9f49fa98d63053a48a77da: stopping tablet replica
I20260926 16:32:40.046464 18722 raft_consensus.cc:2229] T 8b32453a98564720b1f4156961060385 P beada9654a9f49fa98d63053a48a77da [term 1 LEADER]: Raft consensus shutting down.
I20260926 16:32:40.046831 18722 raft_consensus.cc:2258] T 8b32453a98564720b1f4156961060385 P beada9654a9f49fa98d63053a48a77da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:32:40.047026 18596 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b32453a98564720b1f4156961060385 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:32:40 UTC) from {username='oryx', principal='oryx/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:52838
I20260926 16:32:40.047449 18724 tablet_replica.cc:330] T 8b32453a98564720b1f4156961060385 P fe17e60ac69141458c23a427d88a8ca9: stopping tablet replica
I20260926 16:32:40.047761 18724 raft_consensus.cc:2229] T 8b32453a98564720b1f4156961060385 P fe17e60ac69141458c23a427d88a8ca9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:32:40.048089 18724 raft_consensus.cc:2258] T 8b32453a98564720b1f4156961060385 P fe17e60ac69141458c23a427d88a8ca9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:32:40.048417 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 17385
I20260926 16:32:40.050010 18724 ts_tablet_manager.cc:1905] T 8b32453a98564720b1f4156961060385 P fe17e60ac69141458c23a427d88a8ca9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:32:40.050011 18722 ts_tablet_manager.cc:1905] T 8b32453a98564720b1f4156961060385 P beada9654a9f49fa98d63053a48a77da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:32:40.053057 18724 ts_tablet_manager.cc:1918] T 8b32453a98564720b1f4156961060385 P fe17e60ac69141458c23a427d88a8ca9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:32:40.053057 18722 ts_tablet_manager.cc:1918] T 8b32453a98564720b1f4156961060385 P beada9654a9f49fa98d63053a48a77da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:32:40.053164 18724 log.cc:1196] T 8b32453a98564720b1f4156961060385 P fe17e60ac69141458c23a427d88a8ca9: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-2/wal/wals/8b32453a98564720b1f4156961060385
I20260926 16:32:40.053463 18723 tablet_replica.cc:330] T 8b32453a98564720b1f4156961060385 P 452825c51b3246d1acaef2b0bed22348: stopping tablet replica
I20260926 16:32:40.053507 18724 ts_tablet_manager.cc:1939] T 8b32453a98564720b1f4156961060385 P fe17e60ac69141458c23a427d88a8ca9: Deleting consensus metadata
I20260926 16:32:40.053870 18723 raft_consensus.cc:2229] T 8b32453a98564720b1f4156961060385 P 452825c51b3246d1acaef2b0bed22348 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:32:40.054226 18723 raft_consensus.cc:2258] T 8b32453a98564720b1f4156961060385 P 452825c51b3246d1acaef2b0bed22348 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:32:40.054708 18722 log.cc:1196] T 8b32453a98564720b1f4156961060385 P beada9654a9f49fa98d63053a48a77da: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-1/wal/wals/8b32453a98564720b1f4156961060385
I20260926 16:32:40.055056 18722 ts_tablet_manager.cc:1939] T 8b32453a98564720b1f4156961060385 P beada9654a9f49fa98d63053a48a77da: Deleting consensus metadata
I20260926 16:32:40.055974 18723 ts_tablet_manager.cc:1905] T 8b32453a98564720b1f4156961060385 P 452825c51b3246d1acaef2b0bed22348: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 16:32:40.056133 18415 connection.cc:535] server connection from 127.0.0.1:53474 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260926 16:32:40.056118 18552 connection.cc:535] server connection from 127.0.0.1:52838 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 16:32:40.058789 18723 ts_tablet_manager.cc:1918] T 8b32453a98564720b1f4156961060385 P 452825c51b3246d1acaef2b0bed22348: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:32:40.058907 18723 log.cc:1196] T 8b32453a98564720b1f4156961060385 P 452825c51b3246d1acaef2b0bed22348: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-0/wal/wals/8b32453a98564720b1f4156961060385
I20260926 16:32:40.059221 18723 ts_tablet_manager.cc:1939] T 8b32453a98564720b1f4156961060385 P 452825c51b3246d1acaef2b0bed22348: Deleting consensus metadata
W20260926 16:32:40.060284 18275 connection.cc:441] server connection from 127.0.0.1:33416 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33416 (request call id 1) could send its response
I20260926 16:32:40.067217 18675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:40.047869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36512 (local address 127.11.8.253:38689)
0926 16:32:40.047975 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:40.047979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:40.056212 (+ 8233us) server_negotiation.cc:407] Connection header received
0926 16:32:40.056429 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:40.056433 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:40.056498 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:40.056553 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:40.057605 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:40.058207 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:40.059328 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:40.059610 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:40.059715 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:40.061965 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:40.061969 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:40.061972 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:40.061998 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:40.063780 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:40.064258 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:40.064261 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:40.064263 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:40.064341 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:40.064610 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:40.064613 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:40.064615 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:40.065093 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:40.066421 (+ 1328us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:40.066484 (+ 63us) server_negotiation.cc:299] Negotiation successful
0926 16:32:40.066994 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 16:32:40.070399 18141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:40.060148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36528 (local address 127.11.8.253:38689)
0926 16:32:40.060597 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:40.060602 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:40.060615 (+ 13us) server_negotiation.cc:407] Connection header received
0926 16:32:40.060745 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:40.060750 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:40.060808 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:40.060869 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:40.061788 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:40.062524 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:40.063674 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:40.064005 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:40.064070 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:40.065943 (+ 1873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:40.065946 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:40.065949 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:40.065972 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:40.067755 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:40.069099 (+ 1344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:40.069103 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:40.069105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:40.069180 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:40.069472 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:40.069476 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:40.069477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:40.069671 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:40.069766 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:40.070115 (+ 349us) server_negotiation.cc:299] Negotiation successful
0926 16:32:40.070216 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359}
I20260926 16:32:40.071413 18392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:40.060604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53984 (local address 127.11.8.252:40855)
0926 16:32:40.060716 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:40.060720 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:40.062288 (+ 1568us) server_negotiation.cc:407] Connection header received
0926 16:32:40.062449 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:40.062453 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:40.062508 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:40.062560 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:40.063361 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:40.064102 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:40.065278 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:40.065683 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:40.065765 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:40.067787 (+ 2022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:40.067790 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:40.067792 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:40.067816 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:40.069561 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:40.070050 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:40.070054 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:40.070056 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:40.070138 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:40.070468 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:40.070473 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:40.070475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:40.070676 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:40.070776 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:40.071145 (+ 369us) server_negotiation.cc:299] Negotiation successful
0926 16:32:40.071262 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 16:32:40.078404 18674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:40.048405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53978 (local address 127.11.8.252:40855)
0926 16:32:40.048771 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:40.048777 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:40.065850 (+ 17073us) server_negotiation.cc:407] Connection header received
0926 16:32:40.065889 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:40.065892 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:40.065935 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:40.066001 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:40.068764 (+ 2763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:40.069602 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:40.073089 (+ 3487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:40.073531 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:40.073634 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:40.075078 (+ 1444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:40.075081 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:40.075084 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:40.075107 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:40.076763 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:40.077216 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:40.077220 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:40.077222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:40.077308 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:40.077559 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:40.077563 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:40.077564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:40.077773 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:40.077906 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:40.078098 (+ 192us) server_negotiation.cc:299] Negotiation successful
0926 16:32:40.078219 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281}
I20260926 16:32:41.033347 18733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:41.032896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.11.8.194:45311)
0926 16:32:41.033162 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:41.033247 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":76,"threads_started":1}
W20260926 16:32:41.033736 18526 heartbeater.cc:644] Failed to heartbeat to 127.11.8.254:32843 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
I20260926 16:32:41.035748 18734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:41.035200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.11.8.193:55375)
0926 16:32:41.035493 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:41.035627 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":67,"threads_started":1}
W20260926 16:32:41.036088 18389 heartbeater.cc:644] Failed to heartbeat to 127.11.8.254:32843 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
I20260926 16:32:41.037523 18735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:41.037114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.11.8.195:59115)
0926 16:32:41.037339 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:41.037410 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":73,"threads_started":1}
W20260926 16:32:41.037809 18663 heartbeater.cc:644] Failed to heartbeat to 127.11.8.254:32843 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
I20260926 16:32:41.545653 18736 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1aa72ae6508448486ff84262f3af952)
I20260926 16:32:41.545815 18737 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1aa72ae6508448486ff84262f3af952)
I20260926 16:32:41.545966 18736 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: false } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 40855 } attrs { promote: false } }
I20260926 16:32:41.546032 18737 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: false } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 40855 } attrs { promote: false } }
I20260926 16:32:41.547046 18737 leader_election.cc:288] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d1aa72ae6508448486ff84262f3af952 (127.11.8.254:32843), 6b832e0a09284005aeec7045d9cffb05 (127.11.8.253:38689)
I20260926 16:32:41.547047 18736 leader_election.cc:288] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d1aa72ae6508448486ff84262f3af952 (127.11.8.254:32843), ff1d5a9204f14f58bdac74c8ee80c8c9 (127.11.8.252:40855)
I20260926 16:32:41.547659 18739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:41.547163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.0.0.1:38936)
0926 16:32:41.547463 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:41.547543 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":79,"threads_started":1}
I20260926 16:32:41.547722 18738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:41.547163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.0.0.1:38938)
0926 16:32:41.547542 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:41.547616 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":62,"threads_started":1}
W20260926 16:32:41.547816 18194 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
W20260926 16:32:41.547860 18004 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
I20260926 16:32:41.548900 18741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:41.548600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.0.0.1:38954)
0926 16:32:41.548738 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:41.548794 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260926 16:32:41.549005 18004 leader_election.cc:334] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d1aa72ae6508448486ff84262f3af952 (127.11.8.254:32843): Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
I20260926 16:32:41.549612 18739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:41.549394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.0.0.1:38966)
0926 16:32:41.549502 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:41.549552 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260926 16:32:41.549813 18194 leader_election.cc:334] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d1aa72ae6508448486ff84262f3af952 (127.11.8.254:32843): Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
I20260926 16:32:41.551949 18738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:41.547366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:40855 (local address 127.0.0.1:53988)
0926 16:32:41.548229 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:41.548241 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:41.548315 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:41.548600 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:41.548603 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:41.548620 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:41.548716 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:41.548721 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:41.549770 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:41.549775 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:41.550787 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:41.550797 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:41.551486 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:41.551490 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:41.551633 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:41.551652 (+ 19us) client_negotiation.cc:769] Sending connection context
0926 16:32:41.551745 (+ 93us) client_negotiation.cc:240] Negotiation successful
0926 16:32:41.551820 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":808,"mutex_wait_us":22,"thread_start_us":48,"threads_started":1}
I20260926 16:32:41.552050 18742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:41.547615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36538 (local address 127.11.8.253:38689)
0926 16:32:41.547859 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:41.547864 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:41.548196 (+ 332us) server_negotiation.cc:407] Connection header received
0926 16:32:41.548245 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:41.548250 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:41.548308 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:41.548396 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:41.548847 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:41.549589 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:41.551006 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:41.551483 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:41.551644 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:41.551699 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:41.551815 (+ 116us) server_negotiation.cc:299] Negotiation successful
0926 16:32:41.551883 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":176,"thread_start_us":80,"threads_started":1}
I20260926 16:32:41.552485 18744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:41.547964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53988 (local address 127.11.8.252:40855)
0926 16:32:41.548244 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:41.548252 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:41.548294 (+ 42us) server_negotiation.cc:407] Connection header received
0926 16:32:41.548407 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:41.548410 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:41.548455 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:41.548545 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:41.548930 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:41.549681 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:41.550900 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:41.551367 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:41.551485 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:41.551543 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:41.551767 (+ 224us) server_negotiation.cc:299] Negotiation successful
0926 16:32:41.551834 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":222,"spinlock_wait_cycles":9856,"thread_start_us":75,"threads_started":1}
I20260926 16:32:41.552516 18740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:41.547347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.253:38689 (local address 127.0.0.1:36538)
0926 16:32:41.547977 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:41.547990 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:41.548105 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:41.548535 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:41.548540 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:41.548575 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:41.548702 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:41.548709 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:41.549705 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:41.549711 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:41.550907 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:41.550915 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:41.551602 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:41.551608 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:41.551695 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:41.551723 (+ 28us) client_negotiation.cc:769] Sending connection context
0926 16:32:41.551800 (+ 77us) client_negotiation.cc:240] Negotiation successful
0926 16:32:41.551882 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"mutex_wait_us":1,"spinlock_wait_cycles":3584,"thread_start_us":62,"threads_started":1}
I20260926 16:32:41.552888 18219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6b832e0a09284005aeec7045d9cffb05" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" is_pre_election: true
I20260926 16:32:41.552934 18029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6b832e0a09284005aeec7045d9cffb05" is_pre_election: true
I20260926 16:32:41.553102 18219 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b832e0a09284005aeec7045d9cffb05 in term 1.
I20260926 16:32:41.553117 18029 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff1d5a9204f14f58bdac74c8ee80c8c9 in term 1.
I20260926 16:32:41.553458 18193 leader_election.cc:302] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b832e0a09284005aeec7045d9cffb05, ff1d5a9204f14f58bdac74c8ee80c8c9; no voters: d1aa72ae6508448486ff84262f3af952
I20260926 16:32:41.553467 18005 leader_election.cc:302] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b832e0a09284005aeec7045d9cffb05, ff1d5a9204f14f58bdac74c8ee80c8c9; no voters: d1aa72ae6508448486ff84262f3af952
I20260926 16:32:41.553669 18737 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260926 16:32:41.553733 18737 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d1aa72ae6508448486ff84262f3af952)
I20260926 16:32:41.553772 18737 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [term 1 FOLLOWER]: Advancing to term 2
I20260926 16:32:41.553864 18736 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260926 16:32:41.553925 18736 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d1aa72ae6508448486ff84262f3af952)
I20260926 16:32:41.553989 18736 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [term 1 FOLLOWER]: Advancing to term 2
I20260926 16:32:41.555164 18736 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: false } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 40855 } attrs { promote: false } }
I20260926 16:32:41.555164 18737 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: false } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 40855 } attrs { promote: false } }
I20260926 16:32:41.555673 18737 leader_election.cc:288] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [CANDIDATE]: Term 2 election: Requested vote from peers d1aa72ae6508448486ff84262f3af952 (127.11.8.254:32843), 6b832e0a09284005aeec7045d9cffb05 (127.11.8.253:38689)
I20260926 16:32:41.555673 18736 leader_election.cc:288] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [CANDIDATE]: Term 2 election: Requested vote from peers d1aa72ae6508448486ff84262f3af952 (127.11.8.254:32843), ff1d5a9204f14f58bdac74c8ee80c8c9 (127.11.8.252:40855)
I20260926 16:32:41.556078 18738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:41.555811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.0.0.1:38986)
0926 16:32:41.555943 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:41.556004 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260926 16:32:41.556162 18219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6b832e0a09284005aeec7045d9cffb05" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9"
I20260926 16:32:41.556269 18740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:41.555811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.0.0.1:38980)
0926 16:32:41.556016 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:41.556105 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260926 16:32:41.556334 18219 raft_consensus.cc:2379] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b832e0a09284005aeec7045d9cffb05 in current term 2: Already voted for candidate ff1d5a9204f14f58bdac74c8ee80c8c9 in this term.
I20260926 16:32:41.556324 18029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6b832e0a09284005aeec7045d9cffb05"
I20260926 16:32:41.556479 18029 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ff1d5a9204f14f58bdac74c8ee80c8c9 in current term 2: Already voted for candidate 6b832e0a09284005aeec7045d9cffb05 in this term.
I20260926 16:32:41.556963 18739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:41.556742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.0.0.1:38994)
0926 16:32:41.556857 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:41.556919 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260926 16:32:41.557068 18194 leader_election.cc:334] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d1aa72ae6508448486ff84262f3af952 (127.11.8.254:32843): Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
I20260926 16:32:41.557127 18738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:41.556742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.0.0.1:38996)
0926 16:32:41.556987 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:41.557064 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191}
I20260926 16:32:41.557133 18194 leader_election.cc:302] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff1d5a9204f14f58bdac74c8ee80c8c9; no voters: 6b832e0a09284005aeec7045d9cffb05, d1aa72ae6508448486ff84262f3af952
W20260926 16:32:41.557211 18004 leader_election.cc:334] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d1aa72ae6508448486ff84262f3af952 (127.11.8.254:32843): Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
I20260926 16:32:41.557286 18737 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260926 16:32:41.557303 18004 leader_election.cc:302] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b832e0a09284005aeec7045d9cffb05; no voters: d1aa72ae6508448486ff84262f3af952, ff1d5a9204f14f58bdac74c8ee80c8c9
I20260926 16:32:41.557452 18736 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260926 16:32:42.035018 18746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:42.034559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.11.8.194:35981)
0926 16:32:42.034849 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:42.034920 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":96,"threads_started":1}
I20260926 16:32:42.037220 18747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:42.036745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.11.8.193:41501)
0926 16:32:42.037025 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:42.037095 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":91,"threads_started":1}
I20260926 16:32:42.038771 18748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:42.038406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.11.8.195:59793)
0926 16:32:42.038647 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:42.038696 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":84,"threads_started":1}
I20260926 16:32:43.036408 18749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:43.035949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.11.8.194:49615)
0926 16:32:43.036237 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:43.036302 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":78,"threads_started":1}
I20260926 16:32:43.038793 18750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:43.038380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.11.8.193:48295)
0926 16:32:43.038621 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:43.038684 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":76,"threads_started":1}
I20260926 16:32:43.039793 18751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:43.039417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.11.8.195:34201)
0926 16:32:43.039657 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:43.039706 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":64,"threads_started":1}
I20260926 16:32:43.427322 18752 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:32:43.427489 18752 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: false } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 40855 } attrs { promote: false } }
I20260926 16:32:43.427978 18752 leader_election.cc:288] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d1aa72ae6508448486ff84262f3af952 (127.11.8.254:32843), ff1d5a9204f14f58bdac74c8ee80c8c9 (127.11.8.252:40855)
I20260926 16:32:43.428527 18753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:43.428153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.0.0.1:39260)
0926 16:32:43.428396 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:43.428450 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":66,"threads_started":1}
I20260926 16:32:43.428496 18219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6b832e0a09284005aeec7045d9cffb05" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" is_pre_election: true
I20260926 16:32:43.428658 18219 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b832e0a09284005aeec7045d9cffb05 in term 2.
I20260926 16:32:43.429010 18005 leader_election.cc:302] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b832e0a09284005aeec7045d9cffb05, ff1d5a9204f14f58bdac74c8ee80c8c9; no voters:
I20260926 16:32:43.429200 18752 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260926 16:32:43.429279 18752 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:32:43.429291 18753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:43.429071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.0.0.1:39276)
0926 16:32:43.429197 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:43.429238 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260926 16:32:43.429337 18752 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [term 2 FOLLOWER]: Advancing to term 3
W20260926 16:32:43.429405 18004 leader_election.cc:334] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d1aa72ae6508448486ff84262f3af952 (127.11.8.254:32843): Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
I20260926 16:32:43.430387 18752 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: false } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 40855 } attrs { promote: false } }
I20260926 16:32:43.430727 18752 leader_election.cc:288] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [CANDIDATE]: Term 3 election: Requested vote from peers d1aa72ae6508448486ff84262f3af952 (127.11.8.254:32843), ff1d5a9204f14f58bdac74c8ee80c8c9 (127.11.8.252:40855)
I20260926 16:32:43.431079 18753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:43.430849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.0.0.1:39280)
0926 16:32:43.430972 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:43.431028 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260926 16:32:43.431056 18219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6b832e0a09284005aeec7045d9cffb05" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9"
I20260926 16:32:43.431186 18219 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [term 2 FOLLOWER]: Advancing to term 3
I20260926 16:32:43.431689 18753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:43.431494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.0.0.1:39282)
0926 16:32:43.431591 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:43.431638 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260926 16:32:43.431785 18004 leader_election.cc:334] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer d1aa72ae6508448486ff84262f3af952 (127.11.8.254:32843): Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
I20260926 16:32:43.432214 18219 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b832e0a09284005aeec7045d9cffb05 in term 3.
I20260926 16:32:43.432551 18005 leader_election.cc:302] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b832e0a09284005aeec7045d9cffb05, ff1d5a9204f14f58bdac74c8ee80c8c9; no voters: d1aa72ae6508448486ff84262f3af952
I20260926 16:32:43.432753 18752 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [term 3 FOLLOWER]: Leader election won for term 3
I20260926 16:32:43.433003 18752 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [term 3 LEADER]: Becoming Leader. State: Replica: 6b832e0a09284005aeec7045d9cffb05, State: Running, Role: LEADER
I20260926 16:32:43.433267 18752 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: false } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 40855 } attrs { promote: false } }
I20260926 16:32:43.434679 18755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b832e0a09284005aeec7045d9cffb05. Latest consensus state: current_term: 3 leader_uuid: "6b832e0a09284005aeec7045d9cffb05" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: false } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 40855 } attrs { promote: false } } }
I20260926 16:32:43.434785 18755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:43.435150 18757 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:32:43.437305 18757 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:32:43.437664 18757 catalog_manager.cc:1259] Loaded cluster ID: 2d3a2295b7af482ea4f34e28e17b0044
I20260926 16:32:43.437711 18757 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:32:43.438200 18757 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:32:43.438783 18757 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05: Loaded TSK: 0
I20260926 16:32:43.439441 18757 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:32:43.450443 18018 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36528:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:32:43.453709 18018 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:32:43.456663 18753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:43.456420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.0.0.1:39306)
0926 16:32:43.456534 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:43.456597 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260926 16:32:43.456728 18219 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [term 3 FOLLOWER]: Refusing update from remote peer 6b832e0a09284005aeec7045d9cffb05: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260926 16:32:43.457217 18755 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 40855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260926 16:32:43.457508 18753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:43.457218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.0.0.1:39308)
0926 16:32:43.457364 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:43.457427 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260926 16:32:43.457681 18004 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 -> Peer d1aa72ae6508448486ff84262f3af952 (127.11.8.254:32843): Couldn't send request to peer d1aa72ae6508448486ff84262f3af952. Status: Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 16:32:43.458863 18759 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b832e0a09284005aeec7045d9cffb05. Latest consensus state: current_term: 3 leader_uuid: "6b832e0a09284005aeec7045d9cffb05" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: false } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 40855 } attrs { promote: false } } }
I20260926 16:32:43.459012 18759 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:32:43.460022 18755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "6b832e0a09284005aeec7045d9cffb05" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: false } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 40855 } attrs { promote: false } } }
I20260926 16:32:43.460140 18755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05 [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:43.460433 18759 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "6b832e0a09284005aeec7045d9cffb05" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "d1aa72ae6508448486ff84262f3af952" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32843 } } peers { permanent_uuid: "6b832e0a09284005aeec7045d9cffb05" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 38689 } attrs { promote: false } } peers { permanent_uuid: "ff1d5a9204f14f58bdac74c8ee80c8c9" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 40855 } attrs { promote: false } } }
I20260926 16:32:43.460572 18759 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff1d5a9204f14f58bdac74c8ee80c8c9 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:32:43.470079 18753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:43.465578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:45431 (local address 127.0.0.1:53500)
0926 16:32:43.465691 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:43.465705 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:43.465790 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:43.466248 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:43.466251 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:43.466297 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:43.466412 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:43.466419 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:43.467551 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:43.467554 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:43.468821 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:43.468830 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:43.469701 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:43.469704 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:43.469807 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:43.469835 (+ 28us) client_negotiation.cc:769] Sending connection context
0926 16:32:43.469881 (+ 46us) client_negotiation.cc:240] Negotiation successful
0926 16:32:43.469948 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":48}
I20260926 16:32:43.470816 18765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:43.465713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53500 (local address 127.11.8.194:45431)
0926 16:32:43.465985 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:43.465990 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:43.466005 (+ 15us) server_negotiation.cc:407] Connection header received
0926 16:32:43.466079 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:43.466082 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:43.466137 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:43.466229 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:43.466513 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:43.467349 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:43.468944 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:43.469563 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:43.470455 (+ 892us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:43.470557 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:43.470593 (+ 36us) server_negotiation.cc:299] Negotiation successful
0926 16:32:43.470684 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":78,"threads_started":1}
I20260926 16:32:43.471115 18768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:43.466139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:39875 (local address 127.0.0.1:52846)
0926 16:32:43.466905 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:43.466918 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:43.467049 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:43.467392 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:43.467398 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:43.467420 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:43.467537 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:43.467544 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:43.468553 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:43.468556 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:43.469689 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:43.469697 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:43.470693 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:43.470697 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:43.470786 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:43.470807 (+ 21us) client_negotiation.cc:769] Sending connection context
0926 16:32:43.470887 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 16:32:43.470969 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":710,"mutex_wait_us":5,"spinlock_wait_cycles":1408,"thread_start_us":83,"threads_started":1}
I20260926 16:32:43.471299 18767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:43.466230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52846 (local address 127.11.8.195:39875)
0926 16:32:43.466503 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:43.466507 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:43.466987 (+ 480us) server_negotiation.cc:407] Connection header received
0926 16:32:43.467161 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:43.467164 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:43.467211 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:43.467288 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:43.467646 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:43.468443 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:43.469811 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:43.470474 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:43.470547 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:43.470608 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:43.471079 (+ 471us) server_negotiation.cc:299] Negotiation successful
0926 16:32:43.471164 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":90,"threads_started":1}
I20260926 16:32:43.471381 18459 tablet_service.cc:1467] Processing CreateTablet for tablet ac82232bef844923bfd7d22d4e00cdc9 (DEFAULT_TABLE table=test-table [id=863d2b492dfc4066bf007fdba393093e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:43.471700 18459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac82232bef844923bfd7d22d4e00cdc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:43.471836 18596 tablet_service.cc:1467] Processing CreateTablet for tablet ac82232bef844923bfd7d22d4e00cdc9 (DEFAULT_TABLE table=test-table [id=863d2b492dfc4066bf007fdba393093e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:43.472105 18596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac82232bef844923bfd7d22d4e00cdc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:43.474032 18770 tablet_bootstrap.cc:492] T ac82232bef844923bfd7d22d4e00cdc9 P fe17e60ac69141458c23a427d88a8ca9: Bootstrap starting.
I20260926 16:32:43.474167 18771 tablet_bootstrap.cc:492] T ac82232bef844923bfd7d22d4e00cdc9 P beada9654a9f49fa98d63053a48a77da: Bootstrap starting.
I20260926 16:32:43.474994 18770 tablet_bootstrap.cc:654] T ac82232bef844923bfd7d22d4e00cdc9 P fe17e60ac69141458c23a427d88a8ca9: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:43.475128 18771 tablet_bootstrap.cc:654] T ac82232bef844923bfd7d22d4e00cdc9 P beada9654a9f49fa98d63053a48a77da: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:43.475747 18766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:43.466699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:42459 (local address 127.0.0.1:33422)
0926 16:32:43.467322 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:43.467334 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:43.467462 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:43.468978 (+ 1516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:43.468981 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:43.468996 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:43.469075 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:43.469080 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:43.470006 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:43.470009 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:43.470907 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:43.470917 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:43.475299 (+ 4382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:43.475302 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:43.475412 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:43.475436 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 16:32:43.475526 (+ 90us) client_negotiation.cc:240] Negotiation successful
0926 16:32:43.475624 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":88,"threads_started":1}
I20260926 16:32:43.476157 18769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:43.466822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33422 (local address 127.11.8.193:42459)
0926 16:32:43.467052 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:43.467056 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:43.467392 (+ 336us) server_negotiation.cc:407] Connection header received
0926 16:32:43.468822 (+ 1430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:43.468825 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:43.468889 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:43.468974 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:43.469183 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:43.469882 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:43.474309 (+ 4427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:43.475026 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:43.475120 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:43.475198 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:43.475930 (+ 732us) server_negotiation.cc:299] Negotiation successful
0926 16:32:43.476027 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":60,"threads_started":1}
I20260926 16:32:43.476287 18770 tablet_bootstrap.cc:492] T ac82232bef844923bfd7d22d4e00cdc9 P fe17e60ac69141458c23a427d88a8ca9: No bootstrap required, opened a new log
I20260926 16:32:43.476392 18770 ts_tablet_manager.cc:1397] T ac82232bef844923bfd7d22d4e00cdc9 P fe17e60ac69141458c23a427d88a8ca9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 16:32:43.476876 18770 raft_consensus.cc:348] T ac82232bef844923bfd7d22d4e00cdc9 P fe17e60ac69141458c23a427d88a8ca9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beada9654a9f49fa98d63053a48a77da" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45431 } } peers { permanent_uuid: "452825c51b3246d1acaef2b0bed22348" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42459 } } peers { permanent_uuid: "fe17e60ac69141458c23a427d88a8ca9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 39875 } }
I20260926 16:32:43.476987 18770 raft_consensus.cc:374] T ac82232bef844923bfd7d22d4e00cdc9 P fe17e60ac69141458c23a427d88a8ca9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:43.477010 18770 raft_consensus.cc:729] T ac82232bef844923bfd7d22d4e00cdc9 P fe17e60ac69141458c23a427d88a8ca9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe17e60ac69141458c23a427d88a8ca9, State: Initialized, Role: FOLLOWER
I20260926 16:32:43.477164 18770 consensus_queue.cc:260] T ac82232bef844923bfd7d22d4e00cdc9 P fe17e60ac69141458c23a427d88a8ca9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beada9654a9f49fa98d63053a48a77da" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45431 } } peers { permanent_uuid: "452825c51b3246d1acaef2b0bed22348" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42459 } } peers { permanent_uuid: "fe17e60ac69141458c23a427d88a8ca9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 39875 } }
I20260926 16:32:43.477526 18770 ts_tablet_manager.cc:1428] T ac82232bef844923bfd7d22d4e00cdc9 P fe17e60ac69141458c23a427d88a8ca9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 16:32:43.478406 18751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:43.478117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.11.8.195:45863)
0926 16:32:43.478247 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:43.478334 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260926 16:32:43.478945 18322 tablet_service.cc:1467] Processing CreateTablet for tablet ac82232bef844923bfd7d22d4e00cdc9 (DEFAULT_TABLE table=test-table [id=863d2b492dfc4066bf007fdba393093e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:43.479259 18322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac82232bef844923bfd7d22d4e00cdc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:43.480465 18771 tablet_bootstrap.cc:492] T ac82232bef844923bfd7d22d4e00cdc9 P beada9654a9f49fa98d63053a48a77da: No bootstrap required, opened a new log
I20260926 16:32:43.480581 18771 ts_tablet_manager.cc:1397] T ac82232bef844923bfd7d22d4e00cdc9 P beada9654a9f49fa98d63053a48a77da: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260926 16:32:43.481060 18771 raft_consensus.cc:348] T ac82232bef844923bfd7d22d4e00cdc9 P beada9654a9f49fa98d63053a48a77da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beada9654a9f49fa98d63053a48a77da" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45431 } } peers { permanent_uuid: "452825c51b3246d1acaef2b0bed22348" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42459 } } peers { permanent_uuid: "fe17e60ac69141458c23a427d88a8ca9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 39875 } }
I20260926 16:32:43.481165 18771 raft_consensus.cc:374] T ac82232bef844923bfd7d22d4e00cdc9 P beada9654a9f49fa98d63053a48a77da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:43.481194 18771 raft_consensus.cc:729] T ac82232bef844923bfd7d22d4e00cdc9 P beada9654a9f49fa98d63053a48a77da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: beada9654a9f49fa98d63053a48a77da, State: Initialized, Role: FOLLOWER
I20260926 16:32:43.481313 18771 consensus_queue.cc:260] T ac82232bef844923bfd7d22d4e00cdc9 P beada9654a9f49fa98d63053a48a77da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beada9654a9f49fa98d63053a48a77da" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45431 } } peers { permanent_uuid: "452825c51b3246d1acaef2b0bed22348" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42459 } } peers { permanent_uuid: "fe17e60ac69141458c23a427d88a8ca9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 39875 } }
I20260926 16:32:43.481757 18771 ts_tablet_manager.cc:1428] T ac82232bef844923bfd7d22d4e00cdc9 P beada9654a9f49fa98d63053a48a77da: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260926 16:32:43.482411 18664 heartbeater.cc:503] Master 127.11.8.253:38689 was elected leader, sending a full tablet report...
I20260926 16:32:43.482429 18749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:43.482083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.11.8.194:41285)
0926 16:32:43.482289 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:43.482353 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260926 16:32:43.483036 18749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:43.482844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.11.8.194:38473)
0926 16:32:43.482942 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:43.482986 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260926 16:32:43.483251 18776 tablet_bootstrap.cc:492] T ac82232bef844923bfd7d22d4e00cdc9 P 452825c51b3246d1acaef2b0bed22348: Bootstrap starting.
I20260926 16:32:43.483526 18527 heartbeater.cc:503] Master 127.11.8.253:38689 was elected leader, sending a full tablet report...
I20260926 16:32:43.483894 18751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:43.482467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.11.8.195:38403)
0926 16:32:43.483768 (+ 1301us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:43.483823 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1237}
I20260926 16:32:43.485411 18776 tablet_bootstrap.cc:654] T ac82232bef844923bfd7d22d4e00cdc9 P 452825c51b3246d1acaef2b0bed22348: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:43.486713 18776 tablet_bootstrap.cc:492] T ac82232bef844923bfd7d22d4e00cdc9 P 452825c51b3246d1acaef2b0bed22348: No bootstrap required, opened a new log
I20260926 16:32:43.486784 18776 ts_tablet_manager.cc:1397] T ac82232bef844923bfd7d22d4e00cdc9 P 452825c51b3246d1acaef2b0bed22348: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260926 16:32:43.487121 18776 raft_consensus.cc:348] T ac82232bef844923bfd7d22d4e00cdc9 P 452825c51b3246d1acaef2b0bed22348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beada9654a9f49fa98d63053a48a77da" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45431 } } peers { permanent_uuid: "452825c51b3246d1acaef2b0bed22348" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42459 } } peers { permanent_uuid: "fe17e60ac69141458c23a427d88a8ca9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 39875 } }
I20260926 16:32:43.487205 18776 raft_consensus.cc:374] T ac82232bef844923bfd7d22d4e00cdc9 P 452825c51b3246d1acaef2b0bed22348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:43.487226 18776 raft_consensus.cc:729] T ac82232bef844923bfd7d22d4e00cdc9 P 452825c51b3246d1acaef2b0bed22348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 452825c51b3246d1acaef2b0bed22348, State: Initialized, Role: FOLLOWER
I20260926 16:32:43.487353 18776 consensus_queue.cc:260] T ac82232bef844923bfd7d22d4e00cdc9 P 452825c51b3246d1acaef2b0bed22348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beada9654a9f49fa98d63053a48a77da" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45431 } } peers { permanent_uuid: "452825c51b3246d1acaef2b0bed22348" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42459 } } peers { permanent_uuid: "fe17e60ac69141458c23a427d88a8ca9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 39875 } }
I20260926 16:32:43.487709 18776 ts_tablet_manager.cc:1428] T ac82232bef844923bfd7d22d4e00cdc9 P 452825c51b3246d1acaef2b0bed22348: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 16:32:43.488235 18750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:43.487980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.11.8.193:50459)
0926 16:32:43.488116 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:43.488171 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260926 16:32:43.489331 18390 heartbeater.cc:503] Master 127.11.8.253:38689 was elected leader, sending a full tablet report...
I20260926 16:32:43.489514 18750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:43.489260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.11.8.193:55045)
0926 16:32:43.489403 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:43.489457 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260926 16:32:43.568751 18665 tablet.cc:2369] T ac82232bef844923bfd7d22d4e00cdc9 P fe17e60ac69141458c23a427d88a8ca9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 16:32:43.612052 18528 tablet.cc:2369] T ac82232bef844923bfd7d22d4e00cdc9 P beada9654a9f49fa98d63053a48a77da: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:32:43.667918 18774 raft_consensus.cc:482] T ac82232bef844923bfd7d22d4e00cdc9 P fe17e60ac69141458c23a427d88a8ca9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:32:43.668067 18774 raft_consensus.cc:504] T ac82232bef844923bfd7d22d4e00cdc9 P fe17e60ac69141458c23a427d88a8ca9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beada9654a9f49fa98d63053a48a77da" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45431 } } peers { permanent_uuid: "452825c51b3246d1acaef2b0bed22348" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42459 } } peers { permanent_uuid: "fe17e60ac69141458c23a427d88a8ca9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 39875 } }
I20260926 16:32:43.669265 18774 leader_election.cc:288] T ac82232bef844923bfd7d22d4e00cdc9 P fe17e60ac69141458c23a427d88a8ca9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers beada9654a9f49fa98d63053a48a77da (127.11.8.194:45431), 452825c51b3246d1acaef2b0bed22348 (127.11.8.193:42459)
I20260926 16:32:43.672880 18751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:43.669460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:45431 (local address 127.11.8.195:38035)
0926 16:32:43.669627 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:43.669642 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:43.669748 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:43.670040 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:43.670043 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:43.670063 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:43.670184 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:43.670189 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:43.670950 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:43.670955 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:43.671790 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:43.671799 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:43.672454 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:43.672458 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:43.672557 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:43.672585 (+ 28us) client_negotiation.cc:769] Sending connection context
0926 16:32:43.672682 (+ 97us) client_negotiation.cc:240] Negotiation successful
0926 16:32:43.672752 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260926 16:32:43.672880 18765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:43.669680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:38035 (local address 127.11.8.194:45431)
0926 16:32:43.669811 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:43.669817 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:43.669833 (+ 16us) server_negotiation.cc:407] Connection header received
0926 16:32:43.669894 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:43.669896 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:43.669936 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:43.670023 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:43.670325 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:43.670828 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:43.671925 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:43.672325 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:43.672463 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:43.672519 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:43.672690 (+ 171us) server_negotiation.cc:299] Negotiation successful
0926 16:32:43.672757 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 16:32:43.673141 18769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:43.669666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:33643 (local address 127.11.8.193:42459)
0926 16:32:43.669792 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:43.669797 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:43.670083 (+ 286us) server_negotiation.cc:407] Connection header received
0926 16:32:43.670258 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:43.670282 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:43.670342 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:43.670427 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:43.670661 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:43.671154 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:43.672177 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:43.672571 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:43.672696 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:43.672743 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:43.672923 (+ 180us) server_negotiation.cc:299] Negotiation successful
0926 16:32:43.673016 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 16:32:43.673449 18479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac82232bef844923bfd7d22d4e00cdc9" candidate_uuid: "fe17e60ac69141458c23a427d88a8ca9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "beada9654a9f49fa98d63053a48a77da" is_pre_election: true
I20260926 16:32:43.673681 18779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:43.669605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:42459 (local address 127.11.8.195:33643)
0926 16:32:43.669947 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:43.669986 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:43.670122 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:43.670437 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:43.670439 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:43.670454 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:43.670558 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:43.670563 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:43.671267 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:43.671270 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:43.672046 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:43.672054 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:43.672691 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:43.672694 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:43.672793 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:43.672818 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 16:32:43.672909 (+ 91us) client_negotiation.cc:240] Negotiation successful
0926 16:32:43.673505 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"mutex_wait_us":2,"thread_start_us":124,"threads_started":1}
I20260926 16:32:43.673691 18479 raft_consensus.cc:2454] T ac82232bef844923bfd7d22d4e00cdc9 P beada9654a9f49fa98d63053a48a77da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe17e60ac69141458c23a427d88a8ca9 in term 0.
I20260926 16:32:43.674134 18550 leader_election.cc:302] T ac82232bef844923bfd7d22d4e00cdc9 P fe17e60ac69141458c23a427d88a8ca9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: beada9654a9f49fa98d63053a48a77da, fe17e60ac69141458c23a427d88a8ca9; no voters:
I20260926 16:32:43.674220 18342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac82232bef844923bfd7d22d4e00cdc9" candidate_uuid: "fe17e60ac69141458c23a427d88a8ca9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452825c51b3246d1acaef2b0bed22348" is_pre_election: true
I20260926 16:32:43.674432 18342 raft_consensus.cc:2454] T ac82232bef844923bfd7d22d4e00cdc9 P 452825c51b3246d1acaef2b0bed22348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe17e60ac69141458c23a427d88a8ca9 in term 0.
I20260926 16:32:43.674467 18774 raft_consensus.cc:2781] T ac82232bef844923bfd7d22d4e00cdc9 P fe17e60ac69141458c23a427d88a8ca9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:32:43.674520 18774 raft_consensus.cc:482] T ac82232bef844923bfd7d22d4e00cdc9 P fe17e60ac69141458c23a427d88a8ca9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:32:43.674556 18774 raft_consensus.cc:3037] T ac82232bef844923bfd7d22d4e00cdc9 P fe17e60ac69141458c23a427d88a8ca9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:43.675627 18774 raft_consensus.cc:504] T ac82232bef844923bfd7d22d4e00cdc9 P fe17e60ac69141458c23a427d88a8ca9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beada9654a9f49fa98d63053a48a77da" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45431 } } peers { permanent_uuid: "452825c51b3246d1acaef2b0bed22348" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42459 } } peers { permanent_uuid: "fe17e60ac69141458c23a427d88a8ca9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 39875 } }
I20260926 16:32:43.675961 18774 leader_election.cc:288] T ac82232bef844923bfd7d22d4e00cdc9 P fe17e60ac69141458c23a427d88a8ca9 [CANDIDATE]: Term 1 election: Requested vote from peers beada9654a9f49fa98d63053a48a77da (127.11.8.194:45431), 452825c51b3246d1acaef2b0bed22348 (127.11.8.193:42459)
I20260926 16:32:43.676295 18342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac82232bef844923bfd7d22d4e00cdc9" candidate_uuid: "fe17e60ac69141458c23a427d88a8ca9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "452825c51b3246d1acaef2b0bed22348"
I20260926 16:32:43.676331 18479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac82232bef844923bfd7d22d4e00cdc9" candidate_uuid: "fe17e60ac69141458c23a427d88a8ca9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "beada9654a9f49fa98d63053a48a77da"
I20260926 16:32:43.676424 18342 raft_consensus.cc:3037] T ac82232bef844923bfd7d22d4e00cdc9 P 452825c51b3246d1acaef2b0bed22348 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:43.676471 18479 raft_consensus.cc:3037] T ac82232bef844923bfd7d22d4e00cdc9 P beada9654a9f49fa98d63053a48a77da [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:43.677347 18342 raft_consensus.cc:2454] T ac82232bef844923bfd7d22d4e00cdc9 P 452825c51b3246d1acaef2b0bed22348 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe17e60ac69141458c23a427d88a8ca9 in term 1.
I20260926 16:32:43.677634 18552 leader_election.cc:302] T ac82232bef844923bfd7d22d4e00cdc9 P fe17e60ac69141458c23a427d88a8ca9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 452825c51b3246d1acaef2b0bed22348, fe17e60ac69141458c23a427d88a8ca9; no voters:
I20260926 16:32:43.677652 18479 raft_consensus.cc:2454] T ac82232bef844923bfd7d22d4e00cdc9 P beada9654a9f49fa98d63053a48a77da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe17e60ac69141458c23a427d88a8ca9 in term 1.
I20260926 16:32:43.677827 18774 raft_consensus.cc:2781] T ac82232bef844923bfd7d22d4e00cdc9 P fe17e60ac69141458c23a427d88a8ca9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:32:43.678077 18774 raft_consensus.cc:686] T ac82232bef844923bfd7d22d4e00cdc9 P fe17e60ac69141458c23a427d88a8ca9 [term 1 LEADER]: Becoming Leader. State: Replica: fe17e60ac69141458c23a427d88a8ca9, State: Running, Role: LEADER
I20260926 16:32:43.678376 18774 consensus_queue.cc:237] T ac82232bef844923bfd7d22d4e00cdc9 P fe17e60ac69141458c23a427d88a8ca9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beada9654a9f49fa98d63053a48a77da" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45431 } } peers { permanent_uuid: "452825c51b3246d1acaef2b0bed22348" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42459 } } peers { permanent_uuid: "fe17e60ac69141458c23a427d88a8ca9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 39875 } }
I20260926 16:32:43.678874 18779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:43.678600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.11.8.195:44061)
0926 16:32:43.678726 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:43.678786 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260926 16:32:43.681300 18017 catalog_manager.cc:5495] T ac82232bef844923bfd7d22d4e00cdc9 P fe17e60ac69141458c23a427d88a8ca9 reported cstate change: term changed from 0 to 1, leader changed from <none> to fe17e60ac69141458c23a427d88a8ca9 (127.11.8.195). New cstate: current_term: 1 leader_uuid: "fe17e60ac69141458c23a427d88a8ca9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beada9654a9f49fa98d63053a48a77da" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "452825c51b3246d1acaef2b0bed22348" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fe17e60ac69141458c23a427d88a8ca9" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 39875 } health_report { overall_health: HEALTHY } } }
I20260926 16:32:43.708377 18767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:43.704888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52856 (local address 127.11.8.195:39875)
0926 16:32:43.705029 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:43.705034 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:43.705049 (+ 15us) server_negotiation.cc:407] Connection header received
0926 16:32:43.705156 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:43.705159 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:43.705214 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:43.705324 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:32:43.705585 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:43.706091 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:43.706963 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:43.707187 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:43.707295 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:43.707572 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:32:43.707995 (+ 423us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:32:43.708117 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:43.708185 (+ 68us) server_negotiation.cc:299] Negotiation successful
0926 16:32:43.708237 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 16:32:43.711217 18342 raft_consensus.cc:1261] T ac82232bef844923bfd7d22d4e00cdc9 P 452825c51b3246d1acaef2b0bed22348 [term 1 FOLLOWER]: Refusing update from remote peer fe17e60ac69141458c23a427d88a8ca9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:32:43.711459 18479 raft_consensus.cc:1261] T ac82232bef844923bfd7d22d4e00cdc9 P beada9654a9f49fa98d63053a48a77da [term 1 FOLLOWER]: Refusing update from remote peer fe17e60ac69141458c23a427d88a8ca9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:32:43.711740 18780 consensus_queue.cc:1035] T ac82232bef844923bfd7d22d4e00cdc9 P fe17e60ac69141458c23a427d88a8ca9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "452825c51b3246d1acaef2b0bed22348" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:32:43.712001 18774 consensus_queue.cc:1035] T ac82232bef844923bfd7d22d4e00cdc9 P fe17e60ac69141458c23a427d88a8ca9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "beada9654a9f49fa98d63053a48a77da" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:32:43.714092 18749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:43.713799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.11.8.194:53705)
0926 16:32:43.713958 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:43.714016 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260926 16:32:43.714092 18750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:43.713799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.11.8.193:33363)
0926 16:32:43.713958 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:43.714016 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260926 16:32:43.717767 18785 mvcc.cc:205] Tried to move back new op lower bound from 7333643729755660288 to 7333643729629786112. Current Snapshot: MvccSnapshot[applied={T|T < 7333643729755660288}]
I20260926 16:32:43.718559 18779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:43.718146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.11.8.195:33329)
0926 16:32:43.718415 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:43.718479 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175}
I20260926 16:32:43.719210 18784 mvcc.cc:205] Tried to move back new op lower bound from 7333643729755660288 to 7333643729629786112. Current Snapshot: MvccSnapshot[applied={T|T < 7333643729755660288}]
I20260926 16:32:43.720407 18779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:43.720047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.11.8.195:59985)
0926 16:32:43.720281 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:43.720335 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165}
I20260926 16:32:43.720880 18749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:43.720636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.11.8.194:35027)
0926 16:32:43.720742 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:43.720802 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260926 16:32:43.721180 18750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:43.720878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:32843 (local address 127.11.8.193:52255)
0926 16:32:43.721073 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:43.721125 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:32843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
I20260926 16:32:43.727988 18018 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36528:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:32:43.728157 18018 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36528:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:32:43.732033 18018 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6b832e0a09284005aeec7045d9cffb05: Sending DeleteTablet for 3 replicas of tablet ac82232bef844923bfd7d22d4e00cdc9
I20260926 16:32:43.732656 18459 tablet_service.cc:1514] Processing DeleteTablet for tablet ac82232bef844923bfd7d22d4e00cdc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:32:43 UTC) from {username='oryx', principal='oryx/127.11.8.253@KRBTEST.COM'} at 127.0.0.1:53500
I20260926 16:32:43.732913 18322 tablet_service.cc:1514] Processing DeleteTablet for tablet ac82232bef844923bfd7d22d4e00cdc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:32:43 UTC) from {username='oryx', principal='oryx/127.11.8.253@KRBTEST.COM'} at 127.0.0.1:33422
I20260926 16:32:43.733294 18793 tablet_replica.cc:330] T ac82232bef844923bfd7d22d4e00cdc9 P beada9654a9f49fa98d63053a48a77da: stopping tablet replica
I20260926 16:32:43.733438 18793 raft_consensus.cc:2229] T ac82232bef844923bfd7d22d4e00cdc9 P beada9654a9f49fa98d63053a48a77da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:32:43.733593 18793 raft_consensus.cc:2258] T ac82232bef844923bfd7d22d4e00cdc9 P beada9654a9f49fa98d63053a48a77da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:32:43.733584 18596 tablet_service.cc:1514] Processing DeleteTablet for tablet ac82232bef844923bfd7d22d4e00cdc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:32:43 UTC) from {username='oryx', principal='oryx/127.11.8.253@KRBTEST.COM'} at 127.0.0.1:52846
I20260926 16:32:43.733603 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 18164
I20260926 16:32:43.733867 18795 tablet_replica.cc:330] T ac82232bef844923bfd7d22d4e00cdc9 P fe17e60ac69141458c23a427d88a8ca9: stopping tablet replica
I20260926 16:32:43.734166 18795 raft_consensus.cc:2229] T ac82232bef844923bfd7d22d4e00cdc9 P fe17e60ac69141458c23a427d88a8ca9 [term 1 LEADER]: Raft consensus shutting down.
I20260926 16:32:43.734519 18795 raft_consensus.cc:2258] T ac82232bef844923bfd7d22d4e00cdc9 P fe17e60ac69141458c23a427d88a8ca9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:32:43.734977 18793 ts_tablet_manager.cc:1905] T ac82232bef844923bfd7d22d4e00cdc9 P beada9654a9f49fa98d63053a48a77da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:32:43.735090 18795 ts_tablet_manager.cc:1905] T ac82232bef844923bfd7d22d4e00cdc9 P fe17e60ac69141458c23a427d88a8ca9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:32:43.737576 18795 ts_tablet_manager.cc:1918] T ac82232bef844923bfd7d22d4e00cdc9 P fe17e60ac69141458c23a427d88a8ca9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:32:43.737576 18793 ts_tablet_manager.cc:1918] T ac82232bef844923bfd7d22d4e00cdc9 P beada9654a9f49fa98d63053a48a77da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:32:43.737658 18795 log.cc:1196] T ac82232bef844923bfd7d22d4e00cdc9 P fe17e60ac69141458c23a427d88a8ca9: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-2/wal/wals/ac82232bef844923bfd7d22d4e00cdc9
I20260926 16:32:43.737658 18793 log.cc:1196] T ac82232bef844923bfd7d22d4e00cdc9 P beada9654a9f49fa98d63053a48a77da: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790440324823152-11299-0/minicluster-data/ts-1/wal/wals/ac82232bef844923bfd7d22d4e00cdc9
I20260926 16:32:43.737936 18795 ts_tablet_manager.cc:1939] T ac82232bef844923bfd7d22d4e00cdc9 P fe17e60ac69141458c23a427d88a8ca9: Deleting consensus metadata
I20260926 16:32:43.737948 18793 ts_tablet_manager.cc:1939] T ac82232bef844923bfd7d22d4e00cdc9 P beada9654a9f49fa98d63053a48a77da: Deleting consensus metadata
I20260926 16:32:43.738618 18004 catalog_manager.cc:4841] TS beada9654a9f49fa98d63053a48a77da (127.11.8.194:45431): tablet ac82232bef844923bfd7d22d4e00cdc9 (table test-table [id=863d2b492dfc4066bf007fdba393093e]) successfully deleted
I20260926 16:32:43.738703 18005 catalog_manager.cc:4841] TS fe17e60ac69141458c23a427d88a8ca9 (127.11.8.195:39875): tablet ac82232bef844923bfd7d22d4e00cdc9 (table test-table [id=863d2b492dfc4066bf007fdba393093e]) successfully deleted
I20260926 16:32:43.743300 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 18396
I20260926 16:32:43.743625 18766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:43.743329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:42459 (local address 127.0.0.1:33428)
0926 16:32:43.743498 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:43.743549 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.193:42459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260926 16:32:43.744887 18006 catalog_manager.cc:4584] TS 452825c51b3246d1acaef2b0bed22348 (127.11.8.193:42459): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ac82232bef844923bfd7d22d4e00cdc9: Network error: Client connection negotiation failed: client connection to 127.11.8.193:42459: connect: Connection refused (error 111)
I20260926 16:32:43.751655 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 18532
I20260926 16:32:43.757381 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 17881
I20260926 16:32:43.763551 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 17991
2026-09-26T16:32:43Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7395 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:32:43 dist-test-slave-2zvq krb5kdc[18798](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:32:43 dist-test-slave-2zvq krb5kdc[18798](info): set up 2 sockets
Sep 26 16:32:43 dist-test-slave-2zvq krb5kdc[18798](info): commencing operation
krb5kdc: starting...
W20260926 16:32:44.808684 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:32:44 dist-test-slave-2zvq krb5kdc[18798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440364, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:32:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:32:44Z Disabled control of system clock
WARNING: no policy specified for oryx/127.11.8.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.8.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.8.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.254.keytab.
Entry for principal oryx/127.11.8.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.254.keytab.
I20260926 16:32:44.930045 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:38267
--webserver_interface=127.11.8.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.254.keytab
--principal=oryx/127.11.8.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:36559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:38267
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:45.040064 18811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:45.040319 18811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:45.040357 18811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:45.044322 18811 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:32:45.044385 18811 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 16:32:45.044402 18811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:45.044420 18811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:45.044435 18811 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:32:45.044450 18811 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:32:45.049626 18811 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:36559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:38267
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.254.keytab
--principal=oryx/127.11.8.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:38267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18811
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:45.050806 18811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:45.051771 18811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:45.060071 18817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:45.060169 18819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:45.060303 18811 server_base.cc:1034] running on GCE node
W20260926 16:32:45.060174 18816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:45.060961 18811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:45.061964 18811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:45.063124 18811 hybrid_clock.cc:648] HybridClock initialized: now 1790440365063075 us; error 62 us; skew 500 ppm
Sep 26 16:32:45 dist-test-slave-2zvq krb5kdc[18798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440365, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:45.065846 18811 init.cc:377] Logged in from keytab as oryx/127.11.8.254@KRBTEST.COM (short username oryx)
I20260926 16:32:45.066766 18811 webserver.cc:460] Webserver started at http://127.11.8.254:38469/ using document root <none> and password file <none>
I20260926 16:32:45.067312 18811 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:45.067373 18811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:45.067601 18811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:45.069537 18811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "507af6a383f0428fab394c8d26d64eaa"
format_stamp: "Formatted at 2026-09-26 16:32:45 on dist-test-slave-2zvq"
server_key: "fb77ecf1a987219382a0fa8e666f2785"
server_key_iv: "de4877cb9140ddb27f9fe05bafe8e651"
server_key_version: "encryptionkey@0"
I20260926 16:32:45.070096 18811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "507af6a383f0428fab394c8d26d64eaa"
format_stamp: "Formatted at 2026-09-26 16:32:45 on dist-test-slave-2zvq"
server_key: "fb77ecf1a987219382a0fa8e666f2785"
server_key_iv: "de4877cb9140ddb27f9fe05bafe8e651"
server_key_version: "encryptionkey@0"
I20260926 16:32:45.073657 18811 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260926 16:32:45.076176 18826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:45.077441 18811 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:32:45.077574 18811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "507af6a383f0428fab394c8d26d64eaa"
format_stamp: "Formatted at 2026-09-26 16:32:45 on dist-test-slave-2zvq"
server_key: "fb77ecf1a987219382a0fa8e666f2785"
server_key_iv: "de4877cb9140ddb27f9fe05bafe8e651"
server_key_version: "encryptionkey@0"
I20260926 16:32:45.077674 18811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:45.099112 18811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:45.102149 18811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:45.102530 18811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:45.111240 18811 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:38267
I20260926 16:32:45.111258 18888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:38267 every 8 connection(s)
I20260926 16:32:45.112246 18811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:32:45.115077 18889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:45.116559 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 18811
I20260926 16:32:45.116684 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:32:45.116981 11299 external_mini_cluster.cc:1428] Setting key d15dc6db83ad0bb9a88ad0a44c450daf
I20260926 16:32:45.121402 18889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 507af6a383f0428fab394c8d26d64eaa: Bootstrap starting.
I20260926 16:32:45.124023 18889 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 507af6a383f0428fab394c8d26d64eaa: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:45.125097 18889 log.cc:824] T 00000000000000000000000000000000 P 507af6a383f0428fab394c8d26d64eaa: Log is configured to *not* fsync() on all Append() calls
Sep 26 16:32:45 dist-test-slave-2zvq krb5kdc[18798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440364, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM
I20260926 16:32:45.127451 18889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 507af6a383f0428fab394c8d26d64eaa: No bootstrap required, opened a new log
I20260926 16:32:45.130626 18889 raft_consensus.cc:348] T 00000000000000000000000000000000 P 507af6a383f0428fab394c8d26d64eaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "507af6a383f0428fab394c8d26d64eaa" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 38267 } }
I20260926 16:32:45.130890 18889 raft_consensus.cc:374] T 00000000000000000000000000000000 P 507af6a383f0428fab394c8d26d64eaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:45.130970 18889 raft_consensus.cc:729] T 00000000000000000000000000000000 P 507af6a383f0428fab394c8d26d64eaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 507af6a383f0428fab394c8d26d64eaa, State: Initialized, Role: FOLLOWER
I20260926 16:32:45.131433 18889 consensus_queue.cc:260] T 00000000000000000000000000000000 P 507af6a383f0428fab394c8d26d64eaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "507af6a383f0428fab394c8d26d64eaa" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 38267 } }
I20260926 16:32:45.131588 18889 raft_consensus.cc:388] T 00000000000000000000000000000000 P 507af6a383f0428fab394c8d26d64eaa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:32:45.131676 18889 raft_consensus.cc:482] T 00000000000000000000000000000000 P 507af6a383f0428fab394c8d26d64eaa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:32:45.131796 18889 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 507af6a383f0428fab394c8d26d64eaa [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:45.132434 18892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:45.118534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45932 (local address 127.11.8.254:38267)
0926 16:32:45.119011 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:45.119026 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:45.119064 (+ 38us) server_negotiation.cc:407] Connection header received
0926 16:32:45.119696 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:45.119713 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:45.120079 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:45.120443 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:45.121274 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:45.122391 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:45.123266 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:45.123756 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:45.123952 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:45.126756 (+ 2804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:45.126761 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:45.126776 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:45.126801 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:45.129139 (+ 2338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:45.129615 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:45.129621 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:45.129634 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:45.129742 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:45.130024 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:45.130028 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:45.130030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:45.130490 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:45.130699 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:45.130953 (+ 254us) server_negotiation.cc:299] Negotiation successful
0926 16:32:45.131425 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":120,"threads_started":1}
I20260926 16:32:45.132840 18889 raft_consensus.cc:504] T 00000000000000000000000000000000 P 507af6a383f0428fab394c8d26d64eaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "507af6a383f0428fab394c8d26d64eaa" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 38267 } }
I20260926 16:32:45.133179 18889 leader_election.cc:302] T 00000000000000000000000000000000 P 507af6a383f0428fab394c8d26d64eaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 507af6a383f0428fab394c8d26d64eaa; no voters:
I20260926 16:32:45.133523 18889 leader_election.cc:288] T 00000000000000000000000000000000 P 507af6a383f0428fab394c8d26d64eaa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:32:45.133621 18894 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 507af6a383f0428fab394c8d26d64eaa [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:32:45.133833 18894 raft_consensus.cc:686] T 00000000000000000000000000000000 P 507af6a383f0428fab394c8d26d64eaa [term 1 LEADER]: Becoming Leader. State: Replica: 507af6a383f0428fab394c8d26d64eaa, State: Running, Role: LEADER
I20260926 16:32:45.134172 18894 consensus_queue.cc:237] T 00000000000000000000000000000000 P 507af6a383f0428fab394c8d26d64eaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "507af6a383f0428fab394c8d26d64eaa" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 38267 } }
I20260926 16:32:45.134855 18889 sys_catalog.cc:564] T 00000000000000000000000000000000 P 507af6a383f0428fab394c8d26d64eaa [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:32:45.135409 18896 sys_catalog.cc:455] T 00000000000000000000000000000000 P 507af6a383f0428fab394c8d26d64eaa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 507af6a383f0428fab394c8d26d64eaa. Latest consensus state: current_term: 1 leader_uuid: "507af6a383f0428fab394c8d26d64eaa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "507af6a383f0428fab394c8d26d64eaa" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 38267 } } }
I20260926 16:32:45.135627 18896 sys_catalog.cc:458] T 00000000000000000000000000000000 P 507af6a383f0428fab394c8d26d64eaa [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:45.135905 18895 sys_catalog.cc:455] T 00000000000000000000000000000000 P 507af6a383f0428fab394c8d26d64eaa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "507af6a383f0428fab394c8d26d64eaa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "507af6a383f0428fab394c8d26d64eaa" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 38267 } } }
I20260926 16:32:45.135998 18895 sys_catalog.cc:458] T 00000000000000000000000000000000 P 507af6a383f0428fab394c8d26d64eaa [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:45.136066 18903 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:32:45.139063 18903 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:32:45.143824 18903 catalog_manager.cc:1347] Generated new cluster ID: a2536d2ef6fa48338b697b29b89c75c4
I20260926 16:32:45.143885 18903 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:32:45.157171 18903 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:32:45.158037 18903 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:32:45.175148 18903 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 507af6a383f0428fab394c8d26d64eaa: Generated new TSK 0
I20260926 16:32:45.176002 18903 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.11.8.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.8.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.8.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.193.keytab.
Entry for principal oryx/127.11.8.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.193.keytab.
I20260926 16:32:45.214443 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.193:0
--local_ip_for_outbound_sockets=127.11.8.193
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:38267
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.193.keytab
--principal=oryx/127.11.8.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:36559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:45.325402 18915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:45.325667 18915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:45.325712 18915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:45.329903 18915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:45.329964 18915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:45.330004 18915 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 16:32:45.330067 18915 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.193
I20260926 16:32:45.335569 18915 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:36559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.193.keytab
--principal=oryx/127.11.8.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.8.254:38267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18915
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.193
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:45.336746 18915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:45.337814 18915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:45.348382 18920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:45.348394 18923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:45.348425 18915 server_base.cc:1034] running on GCE node
W20260926 16:32:45.348382 18921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:45.349045 18915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:45.349740 18915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:45.350934 18915 hybrid_clock.cc:648] HybridClock initialized: now 1790440365350919 us; error 40 us; skew 500 ppm
Sep 26 16:32:45 dist-test-slave-2zvq krb5kdc[18798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440365, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:45.353894 18915 init.cc:377] Logged in from keytab as oryx/127.11.8.193@KRBTEST.COM (short username oryx)
I20260926 16:32:45.354822 18915 webserver.cc:460] Webserver started at http://127.11.8.193:40189/ using document root <none> and password file <none>
I20260926 16:32:45.355379 18915 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:45.355440 18915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:45.355656 18915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:45.357653 18915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-0/data/instance:
uuid: "d23696d3248b42e2a117be2053d13c10"
format_stamp: "Formatted at 2026-09-26 16:32:45 on dist-test-slave-2zvq"
server_key: "4348bae71ac83296d9af8ad3594888c3"
server_key_iv: "a88174f30481a0e4fcfac384c7c9f2d8"
server_key_version: "encryptionkey@0"
I20260926 16:32:45.358219 18915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance:
uuid: "d23696d3248b42e2a117be2053d13c10"
format_stamp: "Formatted at 2026-09-26 16:32:45 on dist-test-slave-2zvq"
server_key: "4348bae71ac83296d9af8ad3594888c3"
server_key_iv: "a88174f30481a0e4fcfac384c7c9f2d8"
server_key_version: "encryptionkey@0"
I20260926 16:32:45.362258 18915 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260926 16:32:45.365064 18930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:45.366456 18915 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:32:45.366580 18915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-0/wal
uuid: "d23696d3248b42e2a117be2053d13c10"
format_stamp: "Formatted at 2026-09-26 16:32:45 on dist-test-slave-2zvq"
server_key: "4348bae71ac83296d9af8ad3594888c3"
server_key_iv: "a88174f30481a0e4fcfac384c7c9f2d8"
server_key_version: "encryptionkey@0"
I20260926 16:32:45.366691 18915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:45.390245 18915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:45.393181 18915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:45.393328 18915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:45.395118 18915 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:45.395183 18915 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:45.395242 18915 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:45.395283 18915 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 16:32:45 dist-test-slave-2zvq krb5kdc[18798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440365, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.193@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM
I20260926 16:32:45.406445 18915 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.193:42877
I20260926 16:32:45.406536 19045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.193:42877 every 8 connection(s)
I20260926 16:32:45.407584 18915 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
I20260926 16:32:45.409425 18892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:45.395490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:46031 (local address 127.11.8.254:38267)
0926 16:32:45.395644 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:45.395648 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:45.396660 (+ 1012us) server_negotiation.cc:407] Connection header received
0926 16:32:45.397699 (+ 1039us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:45.397702 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:45.397760 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:45.397867 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:45.399243 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:45.399915 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:45.401302 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:45.401553 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:45.401670 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:45.405363 (+ 3693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:45.405369 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:45.405373 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:45.405409 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:45.407162 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:45.407781 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:45.407785 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:45.407787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:45.407878 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:45.408285 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:45.408290 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:45.408293 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:45.408557 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:45.408662 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:45.409143 (+ 481us) server_negotiation.cc:299] Negotiation successful
0926 16:32:45.409269 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 16:32:45.410331 18939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:45.396003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:38267 (local address 127.11.8.193:46031)
0926 16:32:45.396508 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:45.396556 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:45.397454 (+ 898us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:45.398062 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:45.398071 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:45.398531 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:45.399081 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:45.399097 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:45.400057 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:45.400066 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:45.401174 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:45.401185 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:45.401717 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:45.401721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:45.401962 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:45.402801 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:45.402823 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:45.405130 (+ 2307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:45.407296 (+ 2166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:45.407301 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:45.407317 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:45.407677 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:45.407977 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:45.407981 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:45.407983 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:45.408157 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:45.408665 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:45.408671 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:45.408909 (+ 238us) client_negotiation.cc:769] Sending connection context
0926 16:32:45.409084 (+ 175us) client_negotiation.cc:240] Negotiation successful
0926 16:32:45.409387 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":152,"threads_started":1}
I20260926 16:32:45.410876 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 18915
I20260926 16:32:45.411022 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance
I20260926 16:32:45.411384 11299 external_mini_cluster.cc:1428] Setting key 696290cd30e218bcf385a0f97362a2e9
I20260926 16:32:45.411945 19046 heartbeater.cc:347] Connected to a master server at 127.11.8.254:38267
I20260926 16:32:45.412101 19046 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:45.412668 19046 heartbeater.cc:511] Master 127.11.8.254:38267 requested a full tablet report, sending...
I20260926 16:32:45.414350 18842 ts_manager.cc:194] Registered new tserver with Master: d23696d3248b42e2a117be2053d13c10 (127.11.8.193:42877)
I20260926 16:32:45.416054 18842 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.8.193@KRBTEST.COM'} at 127.11.8.193:46031
WARNING: no policy specified for oryx/127.11.8.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.8.194@KRBTEST.COM" created.
I20260926 16:32:45.425494 19052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:45.414099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:38267 (local address 127.11.8.193:55189)
0926 16:32:45.418489 (+ 4390us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:45.418502 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:45.418619 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:45.418889 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:45.418892 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:45.419123 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:45.419221 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:45.419227 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:45.419992 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:45.419995 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:45.420555 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:45.420563 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:45.421031 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:45.421034 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:45.421123 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:45.421675 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:45.421687 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:45.422063 (+ 376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:45.423936 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:45.423942 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:45.423944 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:45.424150 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:45.424515 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:45.424518 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:45.424520 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:45.424633 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:45.425067 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:45.425070 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:45.425148 (+ 78us) client_negotiation.cc:769] Sending connection context
0926 16:32:45.425240 (+ 92us) client_negotiation.cc:240] Negotiation successful
0926 16:32:45.425344 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4308,"thread_start_us":97,"threads_started":1}
I20260926 16:32:45.425696 18892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:45.414182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:55189 (local address 127.11.8.254:38267)
0926 16:32:45.414331 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:45.414337 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:45.418577 (+ 4240us) server_negotiation.cc:407] Connection header received
0926 16:32:45.418722 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:45.418725 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:45.418769 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:45.418871 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:45.419342 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:45.419855 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:45.420674 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:45.420913 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:45.421026 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:45.422194 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:45.422198 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:45.422201 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:45.422224 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:45.423790 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:45.424253 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:45.424258 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:45.424260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:45.424414 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:45.424723 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:45.424727 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:45.424728 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:45.424934 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:45.425079 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:45.425325 (+ 246us) server_negotiation.cc:299] Negotiation successful
0926 16:32:45.425460 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.8.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.194.keytab.
Entry for principal oryx/127.11.8.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.194.keytab.
I20260926 16:32:45.441453 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.194:0
--local_ip_for_outbound_sockets=127.11.8.194
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:38267
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.194.keytab
--principal=oryx/127.11.8.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:36559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:45.553234 19055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:45.553467 19055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:45.553508 19055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:45.557538 19055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:45.557592 19055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:45.557629 19055 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 16:32:45.557713 19055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.194
I20260926 16:32:45.562865 19055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:36559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.194.keytab
--principal=oryx/127.11.8.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.8.254:38267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.194
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:45.563959 19055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:45.564998 19055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:45.574819 19063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:45.574823 19060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:45.574855 19061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:45.574908 19055 server_base.cc:1034] running on GCE node
I20260926 16:32:45.575387 19055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:45.576052 19055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:45.577252 19055 hybrid_clock.cc:648] HybridClock initialized: now 1790440365577236 us; error 27 us; skew 500 ppm
Sep 26 16:32:45 dist-test-slave-2zvq krb5kdc[18798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440365, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:45.580025 19055 init.cc:377] Logged in from keytab as oryx/127.11.8.194@KRBTEST.COM (short username oryx)
I20260926 16:32:45.580885 19055 webserver.cc:460] Webserver started at http://127.11.8.194:37971/ using document root <none> and password file <none>
I20260926 16:32:45.581501 19055 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:45.581593 19055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:45.581816 19055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:45.583977 19055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-1/data/instance:
uuid: "3ced2523e6cb40c7a85b38f149ca8e75"
format_stamp: "Formatted at 2026-09-26 16:32:45 on dist-test-slave-2zvq"
server_key: "5c510577dfc1e0013d6cacc054cca9f4"
server_key_iv: "38ec2f03955234baca0d87132c0f86ad"
server_key_version: "encryptionkey@0"
I20260926 16:32:45.584542 19055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance:
uuid: "3ced2523e6cb40c7a85b38f149ca8e75"
format_stamp: "Formatted at 2026-09-26 16:32:45 on dist-test-slave-2zvq"
server_key: "5c510577dfc1e0013d6cacc054cca9f4"
server_key_iv: "38ec2f03955234baca0d87132c0f86ad"
server_key_version: "encryptionkey@0"
I20260926 16:32:45.588212 19055 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260926 16:32:45.590551 19070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:45.591717 19055 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:32:45.591848 19055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-1/wal
uuid: "3ced2523e6cb40c7a85b38f149ca8e75"
format_stamp: "Formatted at 2026-09-26 16:32:45 on dist-test-slave-2zvq"
server_key: "5c510577dfc1e0013d6cacc054cca9f4"
server_key_iv: "38ec2f03955234baca0d87132c0f86ad"
server_key_version: "encryptionkey@0"
I20260926 16:32:45.591951 19055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:45.608906 19055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:45.612409 19055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:45.612614 19055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:45.614149 19055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:45.614219 19055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:45.614310 19055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:45.614365 19055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 16:32:45 dist-test-slave-2zvq krb5kdc[18798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440365, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.194@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM
I20260926 16:32:45.625332 19055 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.194:45413
I20260926 16:32:45.625403 19185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.194:45413 every 8 connection(s)
I20260926 16:32:45.626426 19055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
I20260926 16:32:45.627573 18892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:45.614875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:42985 (local address 127.11.8.254:38267)
0926 16:32:45.614996 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:45.615000 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:45.616064 (+ 1064us) server_negotiation.cc:407] Connection header received
0926 16:32:45.617043 (+ 979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:45.617050 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:45.617120 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:45.617223 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:45.618647 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:45.619319 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:45.620604 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:45.620826 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:45.620942 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:45.623620 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:45.623623 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:45.623626 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:45.623653 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:45.625329 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:45.625888 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:45.625891 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:45.625893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:45.625971 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:45.626390 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:45.626396 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:45.626401 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:45.626623 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:45.626735 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:45.627322 (+ 587us) server_negotiation.cc:299] Negotiation successful
0926 16:32:45.627439 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 16:32:45.627998 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 19055
I20260926 16:32:45.628146 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance
I20260926 16:32:45.628371 19079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:45.615420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:38267 (local address 127.11.8.194:42985)
0926 16:32:45.615890 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:45.615932 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:45.616778 (+ 846us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:45.617459 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:45.617471 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:45.617918 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:45.618500 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:45.618515 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:45.619446 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:45.619449 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:45.620470 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:45.620479 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:45.620997 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:45.621001 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:45.621154 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:45.621811 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:45.621847 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:45.623440 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:45.625462 (+ 2022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:45.625468 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:45.625483 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:45.625786 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:45.626092 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:45.626095 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:45.626097 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:45.626245 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:45.626730 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:45.626736 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:45.627059 (+ 323us) client_negotiation.cc:769] Sending connection context
0926 16:32:45.627254 (+ 195us) client_negotiation.cc:240] Negotiation successful
0926 16:32:45.627512 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":132,"threads_started":1}
I20260926 16:32:45.628476 11299 external_mini_cluster.cc:1428] Setting key 767b2f5df5ebca2b174686ea7ee683de
I20260926 16:32:45.630342 19186 heartbeater.cc:347] Connected to a master server at 127.11.8.254:38267
I20260926 16:32:45.630483 19186 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:45.631009 19186 heartbeater.cc:511] Master 127.11.8.254:38267 requested a full tablet report, sending...
I20260926 16:32:45.632030 18842 ts_manager.cc:194] Registered new tserver with Master: 3ced2523e6cb40c7a85b38f149ca8e75 (127.11.8.194:45413)
I20260926 16:32:45.632809 18842 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.8.194@KRBTEST.COM'} at 127.11.8.194:42985
WARNING: no policy specified for oryx/127.11.8.195@KRBTEST.COM; defaulting to no policy
I20260926 16:32:45.640039 19193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:45.632181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:38267 (local address 127.11.8.194:36979)
0926 16:32:45.632449 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:45.632467 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:45.632587 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:45.632827 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:45.632830 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:45.633036 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:45.633164 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:45.633171 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:45.634245 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:45.634249 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:45.635042 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:45.635051 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:45.635583 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:45.635586 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:45.635676 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:45.636258 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:45.636272 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:45.636681 (+ 409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:45.638471 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:45.638476 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:45.638478 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:45.638681 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:45.638981 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:45.638985 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:45.638987 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:45.639074 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:45.639483 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:45.639486 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:45.639588 (+ 102us) client_negotiation.cc:769] Sending connection context
0926 16:32:45.639694 (+ 106us) client_negotiation.cc:240] Negotiation successful
0926 16:32:45.639835 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":115,"threads_started":1}
I20260926 16:32:45.640069 18892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:45.631956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:36979 (local address 127.11.8.254:38267)
0926 16:32:45.632064 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:45.632070 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:45.632541 (+ 471us) server_negotiation.cc:407] Connection header received
0926 16:32:45.632678 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:45.632681 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:45.632721 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:45.632813 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:45.633284 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:45.634098 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:45.635174 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:45.635470 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:45.635582 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:45.636810 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:45.636814 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:45.636817 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:45.636861 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:45.638361 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:45.638790 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:45.638793 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:45.638795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:45.638890 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:45.639165 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:45.639169 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:45.639171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:45.639384 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:45.639495 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:45.639749 (+ 254us) server_negotiation.cc:299] Negotiation successful
0926 16:32:45.639861 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.8.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.8.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.195.keytab.
Entry for principal oryx/127.11.8.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.195.keytab.
I20260926 16:32:45.659281 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.195:0
--local_ip_for_outbound_sockets=127.11.8.195
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:38267
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.195.keytab
--principal=oryx/127.11.8.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:36559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:45.773360 19195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:45.773600 19195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:45.773646 19195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:45.777618 19195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:45.777675 19195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:45.777712 19195 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 16:32:45.777745 19195 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.195
I20260926 16:32:45.782943 19195 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:36559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.195.keytab
--principal=oryx/127.11.8.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.8.254:38267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.195
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:45.783869 19195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:45.784878 19195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:45.794329 19200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:45.794332 19203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:45.794418 19195 server_base.cc:1034] running on GCE node
W20260926 16:32:45.794389 19201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:45.795133 19195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:45.795811 19195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:45.797032 19195 hybrid_clock.cc:648] HybridClock initialized: now 1790440365796979 us; error 43 us; skew 500 ppm
Sep 26 16:32:45 dist-test-slave-2zvq krb5kdc[18798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440365, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:45.799921 19195 init.cc:377] Logged in from keytab as oryx/127.11.8.195@KRBTEST.COM (short username oryx)
I20260926 16:32:45.800763 19195 webserver.cc:460] Webserver started at http://127.11.8.195:38917/ using document root <none> and password file <none>
I20260926 16:32:45.801330 19195 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:45.801421 19195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:45.801656 19195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:45.803772 19195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-2/data/instance:
uuid: "d0eec39b5e184211bbd04ce1415efd92"
format_stamp: "Formatted at 2026-09-26 16:32:45 on dist-test-slave-2zvq"
server_key: "c19c45e94279990701c61ccabd8b1e3f"
server_key_iv: "e645b014d2f084199cb997883ba12fb8"
server_key_version: "encryptionkey@0"
I20260926 16:32:45.804380 19195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance:
uuid: "d0eec39b5e184211bbd04ce1415efd92"
format_stamp: "Formatted at 2026-09-26 16:32:45 on dist-test-slave-2zvq"
server_key: "c19c45e94279990701c61ccabd8b1e3f"
server_key_iv: "e645b014d2f084199cb997883ba12fb8"
server_key_version: "encryptionkey@0"
I20260926 16:32:45.808115 19195 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 16:32:45.810557 19210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:45.811813 19195 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260926 16:32:45.811939 19195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-2/wal
uuid: "d0eec39b5e184211bbd04ce1415efd92"
format_stamp: "Formatted at 2026-09-26 16:32:45 on dist-test-slave-2zvq"
server_key: "c19c45e94279990701c61ccabd8b1e3f"
server_key_iv: "e645b014d2f084199cb997883ba12fb8"
server_key_version: "encryptionkey@0"
I20260926 16:32:45.812045 19195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:45.825402 19195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:45.828608 19195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:45.828805 19195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:45.830325 19195 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:45.830386 19195 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:45.830443 19195 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:45.830474 19195 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 16:32:45 dist-test-slave-2zvq krb5kdc[18798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440365, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.195@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM
I20260926 16:32:45.841446 19195 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.195:42027
I20260926 16:32:45.841497 19325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.195:42027 every 8 connection(s)
I20260926 16:32:45.842463 19195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
I20260926 16:32:45.845048 18892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:45.831159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:57731 (local address 127.11.8.254:38267)
0926 16:32:45.831281 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:45.831285 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:45.832265 (+ 980us) server_negotiation.cc:407] Connection header received
0926 16:32:45.833378 (+ 1113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:45.833382 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:45.833452 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:45.833565 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:45.834969 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:45.835699 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:45.837534 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:45.837899 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:45.838019 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:45.841057 (+ 3038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:45.841061 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:45.841066 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:45.841097 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:45.842844 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:45.843477 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:45.843481 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:45.843483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:45.843580 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:45.843931 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:45.843935 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:45.843937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:45.844144 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:45.844256 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:45.844750 (+ 494us) server_negotiation.cc:299] Negotiation successful
0926 16:32:45.844875 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 16:32:45.845533 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 19195
I20260926 16:32:45.845659 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance
I20260926 16:32:45.845964 11299 external_mini_cluster.cc:1428] Setting key ebb66fc36853b32d2bec36e097a13415
I20260926 16:32:45.846103 19219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:45.831646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:38267 (local address 127.11.8.195:57731)
0926 16:32:45.832122 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:45.832161 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:45.833100 (+ 939us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:45.833757 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:45.833768 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:45.834193 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:45.834822 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:45.834838 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:45.835822 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:45.835826 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:45.837415 (+ 1589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:45.837431 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:45.838086 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:45.838093 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:45.838359 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:45.838910 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:45.838933 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:45.840854 (+ 1921us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:45.843012 (+ 2158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:45.843018 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:45.843034 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:45.843361 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:45.843679 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:45.843683 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:45.843685 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:45.843830 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:45.844254 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:45.844261 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:45.844517 (+ 256us) client_negotiation.cc:769] Sending connection context
0926 16:32:45.844707 (+ 190us) client_negotiation.cc:240] Negotiation successful
0926 16:32:45.845030 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":130,"threads_started":1}
I20260926 16:32:45.847505 19326 heartbeater.cc:347] Connected to a master server at 127.11.8.254:38267
I20260926 16:32:45.847656 19326 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:45.848286 19326 heartbeater.cc:511] Master 127.11.8.254:38267 requested a full tablet report, sending...
I20260926 16:32:45.849258 18842 ts_manager.cc:194] Registered new tserver with Master: d0eec39b5e184211bbd04ce1415efd92 (127.11.8.195:42027)
I20260926 16:32:45.849979 18842 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.8.195@KRBTEST.COM'} at 127.11.8.195:57731
I20260926 16:32:45.856765 18892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:45.849616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:36813 (local address 127.11.8.254:38267)
0926 16:32:45.849732 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:45.849737 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:45.850477 (+ 740us) server_negotiation.cc:407] Connection header received
0926 16:32:45.850552 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:45.850555 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:45.850603 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:45.850686 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:45.851252 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:45.851760 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:45.852628 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:45.852857 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:45.852940 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:45.854007 (+ 1067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:45.854010 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:45.854012 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:45.854032 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:45.855488 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:45.855858 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:45.855860 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:45.855862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:45.855928 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:45.856155 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:45.856157 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:45.856158 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:45.856301 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:45.856393 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:45.856582 (+ 189us) server_negotiation.cc:299] Negotiation successful
0926 16:32:45.856664 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 16:32:45.856766 19332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:45.849536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:38267 (local address 127.11.8.195:36813)
0926 16:32:45.850388 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:45.850401 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:45.850486 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:45.850724 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:45.850727 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:45.850949 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:45.851134 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:45.851141 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:45.851913 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:45.851917 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:45.852521 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:45.852530 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:45.852983 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:45.852986 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:45.853059 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:45.853537 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:45.853549 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:45.853886 (+ 337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:45.855583 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:45.855586 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:45.855587 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:45.855775 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:45.856008 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:45.856010 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:45.856012 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:45.856083 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:45.856395 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:45.856397 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:45.856473 (+ 76us) client_negotiation.cc:769] Sending connection context
0926 16:32:45.856549 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 16:32:45.856642 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":727,"mutex_wait_us":20,"thread_start_us":101,"threads_started":1}
I20260926 16:32:45.860018 11299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 16:32:45 dist-test-slave-2zvq krb5kdc[18798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440365, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 16:32:45 dist-test-slave-2zvq krb5kdc[18798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:32:45 dist-test-slave-2zvq krb5kdc[18798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
W20260926 16:32:45.883260 18892 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:45.877023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45944 (local address 127.11.8.254:38267)
0926 16:32:45.877146 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:45.877149 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:45.877161 (+ 12us) server_negotiation.cc:407] Connection header received
0926 16:32:45.877198 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:45.877201 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:45.877238 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:45.877307 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:45.878006 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:45.878568 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:45.879377 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:45.879579 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:45.879652 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:45.883088 (+ 3436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45944: BlockingRecv error: recv got EOF from 127.0.0.1:45944 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 26 16:32:45 dist-test-slave-2zvq krb5kdc[18798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440365, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM
I20260926 16:32:45.892601 18892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:45.884891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45952 (local address 127.11.8.254:38267)
0926 16:32:45.885028 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:45.885031 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:45.885102 (+ 71us) server_negotiation.cc:407] Connection header received
0926 16:32:45.885203 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:45.885205 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:45.885242 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:45.885302 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:45.885849 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:45.886443 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:45.887234 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:45.887429 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:45.887501 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:45.889602 (+ 2101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:45.889606 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:45.889608 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:45.889629 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:45.891231 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:45.891641 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:45.891644 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:45.891645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:45.891722 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:45.891951 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:45.891954 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:45.891955 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:45.892116 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:45.892215 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:45.892397 (+ 182us) server_negotiation.cc:299] Negotiation successful
0926 16:32:45.892486 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 16:32:45.895427 18842 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45952:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:32:45.898555 18842 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:32:45.913019 19353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:45.907668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46818 (local address 127.11.8.195:42027)
0926 16:32:45.908076 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:45.908088 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:45.908111 (+ 23us) server_negotiation.cc:407] Connection header received
0926 16:32:45.908243 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:45.908248 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:45.908444 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:45.908594 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:45.908946 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:45.909662 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:45.911612 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:45.912247 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:45.912371 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:45.912536 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:45.912704 (+ 168us) server_negotiation.cc:299] Negotiation successful
0926 16:32:45.912804 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":111,"threads_started":1}
I20260926 16:32:45.913019 19348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:45.907397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:42027 (local address 127.0.0.1:46818)
0926 16:32:45.908029 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:45.908042 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:45.908126 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:45.908696 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:45.908699 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:45.908715 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:45.908837 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:45.908842 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:45.910142 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:45.910146 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:45.911492 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:45.911501 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:45.912371 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:45.912374 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:45.912484 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:45.912526 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 16:32:45.912671 (+ 145us) client_negotiation.cc:240] Negotiation successful
0926 16:32:45.912829 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"mutex_wait_us":161,"thread_start_us":115,"threads_started":1}
I20260926 16:32:45.913023 19349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:45.906662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:42877 (local address 127.0.0.1:54322)
0926 16:32:45.907152 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:45.907198 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:45.907327 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:45.908302 (+ 975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:45.908308 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:45.908329 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:45.908458 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:45.908465 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:45.909926 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:45.909929 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:45.911456 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:45.911467 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:45.912511 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:45.912516 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:45.912610 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:45.912633 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 16:32:45.912683 (+ 50us) client_negotiation.cc:240] Negotiation successful
0926 16:32:45.912833 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":82,"thread_start_us":82,"threads_started":1}
I20260926 16:32:45.913254 19351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:45.906767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:45413 (local address 127.0.0.1:47832)
0926 16:32:45.907877 (+ 1110us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:45.907891 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:45.908021 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:45.908545 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:45.908549 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:45.908567 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:45.908659 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:45.908668 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:45.910043 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:45.910048 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:45.911255 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:45.911265 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:45.912580 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:45.912584 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:45.912659 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:45.912687 (+ 28us) client_negotiation.cc:769] Sending connection context
0926 16:32:45.912799 (+ 112us) client_negotiation.cc:240] Negotiation successful
0926 16:32:45.912851 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1049,"mutex_wait_us":94,"thread_start_us":102,"threads_started":1}
I20260926 16:32:45.913410 19350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:45.906811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54322 (local address 127.11.8.193:42877)
0926 16:32:45.907780 (+ 969us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:45.907798 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:45.907822 (+ 24us) server_negotiation.cc:407] Connection header received
0926 16:32:45.907903 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:45.907913 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:45.908144 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:45.908301 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:45.908610 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:45.909809 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:45.911596 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:45.912367 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:45.912981 (+ 614us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:45.913122 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:45.913176 (+ 54us) server_negotiation.cc:299] Negotiation successful
0926 16:32:45.913263 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":855,"thread_start_us":73,"threads_started":1}
I20260926 16:32:45.913630 19352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:45.907463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47832 (local address 127.11.8.194:45413)
0926 16:32:45.907892 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:45.907903 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:45.908119 (+ 216us) server_negotiation.cc:407] Connection header received
0926 16:32:45.908199 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:45.908207 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:45.908387 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:45.908561 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:45.908870 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:45.909919 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:45.911377 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:45.912255 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:45.912364 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:45.912531 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:45.913387 (+ 856us) server_negotiation.cc:299] Negotiation successful
0926 16:32:45.913493 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":115,"threads_started":1}
I20260926 16:32:45.916234 19260 tablet_service.cc:1467] Processing CreateTablet for tablet 987a731df1e14abea8547dae27aaabd6 (DEFAULT_TABLE table=test-table [id=14ca7e3c49e94d4dad46cec6066cf473]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:45.916234 19120 tablet_service.cc:1467] Processing CreateTablet for tablet 987a731df1e14abea8547dae27aaabd6 (DEFAULT_TABLE table=test-table [id=14ca7e3c49e94d4dad46cec6066cf473]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:45.917164 19260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 987a731df1e14abea8547dae27aaabd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:45.917165 19120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 987a731df1e14abea8547dae27aaabd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:45.920343 18980 tablet_service.cc:1467] Processing CreateTablet for tablet 987a731df1e14abea8547dae27aaabd6 (DEFAULT_TABLE table=test-table [id=14ca7e3c49e94d4dad46cec6066cf473]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:45.921561 18980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 987a731df1e14abea8547dae27aaabd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:45.923918 19354 tablet_bootstrap.cc:492] T 987a731df1e14abea8547dae27aaabd6 P 3ced2523e6cb40c7a85b38f149ca8e75: Bootstrap starting.
I20260926 16:32:45.925549 19355 tablet_bootstrap.cc:492] T 987a731df1e14abea8547dae27aaabd6 P d0eec39b5e184211bbd04ce1415efd92: Bootstrap starting.
I20260926 16:32:45.926222 19354 tablet_bootstrap.cc:654] T 987a731df1e14abea8547dae27aaabd6 P 3ced2523e6cb40c7a85b38f149ca8e75: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:45.926977 19354 log.cc:824] T 987a731df1e14abea8547dae27aaabd6 P 3ced2523e6cb40c7a85b38f149ca8e75: Log is configured to *not* fsync() on all Append() calls
I20260926 16:32:45.927687 19355 tablet_bootstrap.cc:654] T 987a731df1e14abea8547dae27aaabd6 P d0eec39b5e184211bbd04ce1415efd92: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:45.928519 19355 log.cc:824] T 987a731df1e14abea8547dae27aaabd6 P d0eec39b5e184211bbd04ce1415efd92: Log is configured to *not* fsync() on all Append() calls
I20260926 16:32:45.929502 19354 tablet_bootstrap.cc:492] T 987a731df1e14abea8547dae27aaabd6 P 3ced2523e6cb40c7a85b38f149ca8e75: No bootstrap required, opened a new log
I20260926 16:32:45.929664 19357 tablet_bootstrap.cc:492] T 987a731df1e14abea8547dae27aaabd6 P d23696d3248b42e2a117be2053d13c10: Bootstrap starting.
I20260926 16:32:45.929701 19354 ts_tablet_manager.cc:1397] T 987a731df1e14abea8547dae27aaabd6 P 3ced2523e6cb40c7a85b38f149ca8e75: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260926 16:32:45.930733 19355 tablet_bootstrap.cc:492] T 987a731df1e14abea8547dae27aaabd6 P d0eec39b5e184211bbd04ce1415efd92: No bootstrap required, opened a new log
I20260926 16:32:45.930918 19355 ts_tablet_manager.cc:1397] T 987a731df1e14abea8547dae27aaabd6 P d0eec39b5e184211bbd04ce1415efd92: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260926 16:32:45.931567 19357 tablet_bootstrap.cc:654] T 987a731df1e14abea8547dae27aaabd6 P d23696d3248b42e2a117be2053d13c10: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:45.932502 19357 log.cc:824] T 987a731df1e14abea8547dae27aaabd6 P d23696d3248b42e2a117be2053d13c10: Log is configured to *not* fsync() on all Append() calls
I20260926 16:32:45.933665 19354 raft_consensus.cc:348] T 987a731df1e14abea8547dae27aaabd6 P 3ced2523e6cb40c7a85b38f149ca8e75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:45.933926 19354 raft_consensus.cc:374] T 987a731df1e14abea8547dae27aaabd6 P 3ced2523e6cb40c7a85b38f149ca8e75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:45.934028 19354 raft_consensus.cc:729] T 987a731df1e14abea8547dae27aaabd6 P 3ced2523e6cb40c7a85b38f149ca8e75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ced2523e6cb40c7a85b38f149ca8e75, State: Initialized, Role: FOLLOWER
I20260926 16:32:45.934266 19357 tablet_bootstrap.cc:492] T 987a731df1e14abea8547dae27aaabd6 P d23696d3248b42e2a117be2053d13c10: No bootstrap required, opened a new log
I20260926 16:32:45.934507 19357 ts_tablet_manager.cc:1397] T 987a731df1e14abea8547dae27aaabd6 P d23696d3248b42e2a117be2053d13c10: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260926 16:32:45.934515 19354 consensus_queue.cc:260] T 987a731df1e14abea8547dae27aaabd6 P 3ced2523e6cb40c7a85b38f149ca8e75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:45.934705 19355 raft_consensus.cc:348] T 987a731df1e14abea8547dae27aaabd6 P d0eec39b5e184211bbd04ce1415efd92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:45.934978 19355 raft_consensus.cc:374] T 987a731df1e14abea8547dae27aaabd6 P d0eec39b5e184211bbd04ce1415efd92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:45.935060 19355 raft_consensus.cc:729] T 987a731df1e14abea8547dae27aaabd6 P d0eec39b5e184211bbd04ce1415efd92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0eec39b5e184211bbd04ce1415efd92, State: Initialized, Role: FOLLOWER
I20260926 16:32:45.935290 19186 heartbeater.cc:503] Master 127.11.8.254:38267 was elected leader, sending a full tablet report...
I20260926 16:32:45.935559 19354 ts_tablet_manager.cc:1428] T 987a731df1e14abea8547dae27aaabd6 P 3ced2523e6cb40c7a85b38f149ca8e75: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260926 16:32:45.935928 19355 consensus_queue.cc:260] T 987a731df1e14abea8547dae27aaabd6 P d0eec39b5e184211bbd04ce1415efd92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:45.936756 19326 heartbeater.cc:503] Master 127.11.8.254:38267 was elected leader, sending a full tablet report...
I20260926 16:32:45.936878 19355 ts_tablet_manager.cc:1428] T 987a731df1e14abea8547dae27aaabd6 P d0eec39b5e184211bbd04ce1415efd92: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260926 16:32:45.939046 19357 raft_consensus.cc:348] T 987a731df1e14abea8547dae27aaabd6 P d23696d3248b42e2a117be2053d13c10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:45.939307 19357 raft_consensus.cc:374] T 987a731df1e14abea8547dae27aaabd6 P d23696d3248b42e2a117be2053d13c10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:45.939393 19357 raft_consensus.cc:729] T 987a731df1e14abea8547dae27aaabd6 P d23696d3248b42e2a117be2053d13c10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d23696d3248b42e2a117be2053d13c10, State: Initialized, Role: FOLLOWER
I20260926 16:32:45.939801 19357 consensus_queue.cc:260] T 987a731df1e14abea8547dae27aaabd6 P d23696d3248b42e2a117be2053d13c10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:45.940397 19046 heartbeater.cc:503] Master 127.11.8.254:38267 was elected leader, sending a full tablet report...
I20260926 16:32:45.940634 19357 ts_tablet_manager.cc:1428] T 987a731df1e14abea8547dae27aaabd6 P d23696d3248b42e2a117be2053d13c10: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260926 16:32:46.061981 19362 raft_consensus.cc:482] T 987a731df1e14abea8547dae27aaabd6 P d23696d3248b42e2a117be2053d13c10 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:32:46.062237 19362 raft_consensus.cc:504] T 987a731df1e14abea8547dae27aaabd6 P d23696d3248b42e2a117be2053d13c10 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:46.063478 19362 leader_election.cc:288] T 987a731df1e14abea8547dae27aaabd6 P d23696d3248b42e2a117be2053d13c10 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ced2523e6cb40c7a85b38f149ca8e75 (127.11.8.194:45413), d0eec39b5e184211bbd04ce1415efd92 (127.11.8.195:42027)
I20260926 16:32:46.067539 19353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:46.063869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:60319 (local address 127.11.8.195:42027)
0926 16:32:46.064005 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:46.064010 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:46.064227 (+ 217us) server_negotiation.cc:407] Connection header received
0926 16:32:46.064368 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:46.064370 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:46.064412 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:46.064506 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:46.064824 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:46.065312 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:46.066529 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:46.066901 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:46.067027 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:46.067082 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:46.067328 (+ 246us) server_negotiation.cc:299] Negotiation successful
0926 16:32:46.067406 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 16:32:46.067540 19352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:46.063812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:51801 (local address 127.11.8.194:45413)
0926 16:32:46.063943 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:46.063948 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:46.064113 (+ 165us) server_negotiation.cc:407] Connection header received
0926 16:32:46.064322 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:46.064328 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:46.064391 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:46.064450 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:46.064798 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:46.065306 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:46.066492 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:46.066897 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:46.067024 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:46.067086 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:46.067359 (+ 273us) server_negotiation.cc:299] Negotiation successful
0926 16:32:46.067423 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 16:32:46.067569 19364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:46.063792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:42027 (local address 127.11.8.193:60319)
0926 16:32:46.064137 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:46.064148 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:46.064266 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:46.064532 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:46.064535 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:46.064566 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:46.064692 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:46.064697 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:46.065443 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:46.065447 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:46.066389 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:46.066398 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:46.067024 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:46.067028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:46.067193 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:46.067224 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 16:32:46.067282 (+ 58us) client_negotiation.cc:240] Negotiation successful
0926 16:32:46.067399 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":150,"thread_start_us":146,"threads_started":1}
I20260926 16:32:46.067576 19363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:46.063659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:45413 (local address 127.11.8.193:51801)
0926 16:32:46.064023 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:46.064035 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:46.064167 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:46.064502 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:46.064505 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:46.064521 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:46.064682 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:46.064688 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:46.065415 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:46.065419 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:46.066378 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:46.066387 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:46.067017 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:46.067021 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:46.067218 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:46.067237 (+ 19us) client_negotiation.cc:769] Sending connection context
0926 16:32:46.067329 (+ 92us) client_negotiation.cc:240] Negotiation successful
0926 16:32:46.067421 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":128,"thread_start_us":120,"threads_started":1}
I20260926 16:32:46.068292 19280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "987a731df1e14abea8547dae27aaabd6" candidate_uuid: "d23696d3248b42e2a117be2053d13c10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0eec39b5e184211bbd04ce1415efd92" is_pre_election: true
I20260926 16:32:46.068318 19140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "987a731df1e14abea8547dae27aaabd6" candidate_uuid: "d23696d3248b42e2a117be2053d13c10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" is_pre_election: true
I20260926 16:32:46.068588 19280 raft_consensus.cc:2454] T 987a731df1e14abea8547dae27aaabd6 P d0eec39b5e184211bbd04ce1415efd92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d23696d3248b42e2a117be2053d13c10 in term 0.
I20260926 16:32:46.068601 19140 raft_consensus.cc:2454] T 987a731df1e14abea8547dae27aaabd6 P 3ced2523e6cb40c7a85b38f149ca8e75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d23696d3248b42e2a117be2053d13c10 in term 0.
I20260926 16:32:46.069097 18933 leader_election.cc:302] T 987a731df1e14abea8547dae27aaabd6 P d23696d3248b42e2a117be2053d13c10 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3ced2523e6cb40c7a85b38f149ca8e75, d0eec39b5e184211bbd04ce1415efd92, d23696d3248b42e2a117be2053d13c10; no voters:
I20260926 16:32:46.069375 19362 raft_consensus.cc:2781] T 987a731df1e14abea8547dae27aaabd6 P d23696d3248b42e2a117be2053d13c10 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:32:46.069459 19362 raft_consensus.cc:482] T 987a731df1e14abea8547dae27aaabd6 P d23696d3248b42e2a117be2053d13c10 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:32:46.069501 19362 raft_consensus.cc:3037] T 987a731df1e14abea8547dae27aaabd6 P d23696d3248b42e2a117be2053d13c10 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:46.070703 19362 raft_consensus.cc:504] T 987a731df1e14abea8547dae27aaabd6 P d23696d3248b42e2a117be2053d13c10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:46.071012 19362 leader_election.cc:288] T 987a731df1e14abea8547dae27aaabd6 P d23696d3248b42e2a117be2053d13c10 [CANDIDATE]: Term 1 election: Requested vote from peers 3ced2523e6cb40c7a85b38f149ca8e75 (127.11.8.194:45413), d0eec39b5e184211bbd04ce1415efd92 (127.11.8.195:42027)
I20260926 16:32:46.071362 19280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "987a731df1e14abea8547dae27aaabd6" candidate_uuid: "d23696d3248b42e2a117be2053d13c10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0eec39b5e184211bbd04ce1415efd92"
I20260926 16:32:46.071492 19280 raft_consensus.cc:3037] T 987a731df1e14abea8547dae27aaabd6 P d0eec39b5e184211bbd04ce1415efd92 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:46.072521 19280 raft_consensus.cc:2454] T 987a731df1e14abea8547dae27aaabd6 P d0eec39b5e184211bbd04ce1415efd92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d23696d3248b42e2a117be2053d13c10 in term 1.
I20260926 16:32:46.072782 19140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "987a731df1e14abea8547dae27aaabd6" candidate_uuid: "d23696d3248b42e2a117be2053d13c10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ced2523e6cb40c7a85b38f149ca8e75"
I20260926 16:32:46.072836 18933 leader_election.cc:302] T 987a731df1e14abea8547dae27aaabd6 P d23696d3248b42e2a117be2053d13c10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0eec39b5e184211bbd04ce1415efd92, d23696d3248b42e2a117be2053d13c10; no voters:
I20260926 16:32:46.072913 19140 raft_consensus.cc:3037] T 987a731df1e14abea8547dae27aaabd6 P 3ced2523e6cb40c7a85b38f149ca8e75 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:46.073006 19362 raft_consensus.cc:2781] T 987a731df1e14abea8547dae27aaabd6 P d23696d3248b42e2a117be2053d13c10 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:32:46.073213 19362 raft_consensus.cc:686] T 987a731df1e14abea8547dae27aaabd6 P d23696d3248b42e2a117be2053d13c10 [term 1 LEADER]: Becoming Leader. State: Replica: d23696d3248b42e2a117be2053d13c10, State: Running, Role: LEADER
I20260926 16:32:46.073534 19362 consensus_queue.cc:237] T 987a731df1e14abea8547dae27aaabd6 P d23696d3248b42e2a117be2053d13c10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:46.074612 19140 raft_consensus.cc:2454] T 987a731df1e14abea8547dae27aaabd6 P 3ced2523e6cb40c7a85b38f149ca8e75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d23696d3248b42e2a117be2053d13c10 in term 1.
I20260926 16:32:46.076856 18842 catalog_manager.cc:5495] T 987a731df1e14abea8547dae27aaabd6 P d23696d3248b42e2a117be2053d13c10 reported cstate change: term changed from 0 to 1, leader changed from <none> to d23696d3248b42e2a117be2053d13c10 (127.11.8.193). New cstate: current_term: 1 leader_uuid: "d23696d3248b42e2a117be2053d13c10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } health_report { overall_health: UNKNOWN } } }
W20260926 16:32:46.093866 19327 tablet.cc:2369] T 987a731df1e14abea8547dae27aaabd6 P d0eec39b5e184211bbd04ce1415efd92: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:32:46.102890 19350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:46.099653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54336 (local address 127.11.8.193:42877)
0926 16:32:46.099780 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:46.099784 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:46.099798 (+ 14us) server_negotiation.cc:407] Connection header received
0926 16:32:46.099890 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:46.099893 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:46.099946 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:46.100041 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:32:46.100241 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:46.100732 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:46.101538 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:46.101744 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:46.101817 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:46.102059 (+ 242us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:32:46.102559 (+ 500us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:32:46.102646 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:46.102739 (+ 93us) server_negotiation.cc:299] Negotiation successful
0926 16:32:46.102785 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 16:32:46.108486 19140 raft_consensus.cc:1261] T 987a731df1e14abea8547dae27aaabd6 P 3ced2523e6cb40c7a85b38f149ca8e75 [term 1 FOLLOWER]: Refusing update from remote peer d23696d3248b42e2a117be2053d13c10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:32:46.108479 19280 raft_consensus.cc:1261] T 987a731df1e14abea8547dae27aaabd6 P d0eec39b5e184211bbd04ce1415efd92 [term 1 FOLLOWER]: Refusing update from remote peer d23696d3248b42e2a117be2053d13c10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:32:46.109201 19365 consensus_queue.cc:1035] T 987a731df1e14abea8547dae27aaabd6 P d23696d3248b42e2a117be2053d13c10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:32:46.109375 19362 consensus_queue.cc:1035] T 987a731df1e14abea8547dae27aaabd6 P d23696d3248b42e2a117be2053d13c10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:32:46.119949 19368 mvcc.cc:205] Tried to move back new op lower bound from 7333643739573903360 to 7333643739440558080. Current Snapshot: MvccSnapshot[applied={T|T < 7333643739573903360 or (T in {7333643739573903360})}]
I20260926 16:32:46.120685 19370 mvcc.cc:205] Tried to move back new op lower bound from 7333643739573903360 to 7333643739440558080. Current Snapshot: MvccSnapshot[applied={T|T < 7333643739573903360}]
I20260926 16:32:46.122117 19369 mvcc.cc:205] Tried to move back new op lower bound from 7333643739573903360 to 7333643739440558080. Current Snapshot: MvccSnapshot[applied={T|T < 7333643739573903360}]
I20260926 16:32:46.127171 18843 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45952:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:32:46.127362 18843 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45952:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:32:46.130045 18843 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 507af6a383f0428fab394c8d26d64eaa: Sending DeleteTablet for 3 replicas of tablet 987a731df1e14abea8547dae27aaabd6
I20260926 16:32:46.130926 19120 tablet_service.cc:1514] Processing DeleteTablet for tablet 987a731df1e14abea8547dae27aaabd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:32:46 UTC) from {username='oryx', principal='oryx/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:47832
I20260926 16:32:46.130951 19260 tablet_service.cc:1514] Processing DeleteTablet for tablet 987a731df1e14abea8547dae27aaabd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:32:46 UTC) from {username='oryx', principal='oryx/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:46818
I20260926 16:32:46.131397 18980 tablet_service.cc:1514] Processing DeleteTablet for tablet 987a731df1e14abea8547dae27aaabd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:32:46 UTC) from {username='oryx', principal='oryx/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:54322
I20260926 16:32:46.131860 19380 tablet_replica.cc:330] T 987a731df1e14abea8547dae27aaabd6 P d23696d3248b42e2a117be2053d13c10: stopping tablet replica
I20260926 16:32:46.131834 18843 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45952:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 16:32:46.131991 19378 tablet_replica.cc:330] T 987a731df1e14abea8547dae27aaabd6 P 3ced2523e6cb40c7a85b38f149ca8e75: stopping tablet replica
I20260926 16:32:46.132205 19380 raft_consensus.cc:2229] T 987a731df1e14abea8547dae27aaabd6 P d23696d3248b42e2a117be2053d13c10 [term 1 LEADER]: Raft consensus shutting down.
W20260926 16:32:46.132362 18843 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:32:46.132412 19378 raft_consensus.cc:2229] T 987a731df1e14abea8547dae27aaabd6 P 3ced2523e6cb40c7a85b38f149ca8e75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:32:46.132584 19380 raft_consensus.cc:2258] T 987a731df1e14abea8547dae27aaabd6 P d23696d3248b42e2a117be2053d13c10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:32:46.132858 19378 raft_consensus.cc:2258] T 987a731df1e14abea8547dae27aaabd6 P 3ced2523e6cb40c7a85b38f149ca8e75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:32:46.134348 19380 ts_tablet_manager.cc:1905] T 987a731df1e14abea8547dae27aaabd6 P d23696d3248b42e2a117be2053d13c10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:32:46.137053 18980 tablet_service.cc:1467] Processing CreateTablet for tablet 099453fa543c4196b8898a17e1589412 (DEFAULT_TABLE table=test-table [id=ff0dc02d63fc4939a5179d0b50508b29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:46.137323 18980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 099453fa543c4196b8898a17e1589412. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:46.137569 19380 ts_tablet_manager.cc:1918] T 987a731df1e14abea8547dae27aaabd6 P d23696d3248b42e2a117be2053d13c10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:32:46.137614 19378 ts_tablet_manager.cc:1905] T 987a731df1e14abea8547dae27aaabd6 P 3ced2523e6cb40c7a85b38f149ca8e75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:32:46.137662 19380 log.cc:1196] T 987a731df1e14abea8547dae27aaabd6 P d23696d3248b42e2a117be2053d13c10: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-0/wal/wals/987a731df1e14abea8547dae27aaabd6
I20260926 16:32:46.138003 19380 ts_tablet_manager.cc:1939] T 987a731df1e14abea8547dae27aaabd6 P d23696d3248b42e2a117be2053d13c10: Deleting consensus metadata
I20260926 16:32:46.138473 19379 tablet_replica.cc:330] T 987a731df1e14abea8547dae27aaabd6 P d0eec39b5e184211bbd04ce1415efd92: stopping tablet replica
I20260926 16:32:46.138768 19379 raft_consensus.cc:2229] T 987a731df1e14abea8547dae27aaabd6 P d0eec39b5e184211bbd04ce1415efd92 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:32:46.139077 19379 raft_consensus.cc:2258] T 987a731df1e14abea8547dae27aaabd6 P d0eec39b5e184211bbd04ce1415efd92 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:32:46.139678 19357 tablet_bootstrap.cc:492] T 099453fa543c4196b8898a17e1589412 P d23696d3248b42e2a117be2053d13c10: Bootstrap starting.
I20260926 16:32:46.140657 19357 tablet_bootstrap.cc:654] T 099453fa543c4196b8898a17e1589412 P d23696d3248b42e2a117be2053d13c10: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:46.140731 19378 ts_tablet_manager.cc:1918] T 987a731df1e14abea8547dae27aaabd6 P 3ced2523e6cb40c7a85b38f149ca8e75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:32:46.140825 19378 log.cc:1196] T 987a731df1e14abea8547dae27aaabd6 P 3ced2523e6cb40c7a85b38f149ca8e75: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-1/wal/wals/987a731df1e14abea8547dae27aaabd6
I20260926 16:32:46.141171 19378 ts_tablet_manager.cc:1939] T 987a731df1e14abea8547dae27aaabd6 P 3ced2523e6cb40c7a85b38f149ca8e75: Deleting consensus metadata
I20260926 16:32:46.141600 18829 catalog_manager.cc:4841] TS d23696d3248b42e2a117be2053d13c10 (127.11.8.193:42877): tablet 987a731df1e14abea8547dae27aaabd6 (table test-table [id=14ca7e3c49e94d4dad46cec6066cf473]) successfully deleted
I20260926 16:32:46.141916 19260 tablet_service.cc:1467] Processing CreateTablet for tablet 099453fa543c4196b8898a17e1589412 (DEFAULT_TABLE table=test-table [id=ff0dc02d63fc4939a5179d0b50508b29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:46.142191 19260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 099453fa543c4196b8898a17e1589412. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:46.142162 19120 tablet_service.cc:1467] Processing CreateTablet for tablet 099453fa543c4196b8898a17e1589412 (DEFAULT_TABLE table=test-table [id=ff0dc02d63fc4939a5179d0b50508b29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:46.142405 19120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 099453fa543c4196b8898a17e1589412. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:46.142789 19357 tablet_bootstrap.cc:492] T 099453fa543c4196b8898a17e1589412 P d23696d3248b42e2a117be2053d13c10: No bootstrap required, opened a new log
I20260926 16:32:46.142854 19357 ts_tablet_manager.cc:1397] T 099453fa543c4196b8898a17e1589412 P d23696d3248b42e2a117be2053d13c10: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260926 16:32:46.143065 19379 ts_tablet_manager.cc:1905] T 987a731df1e14abea8547dae27aaabd6 P d0eec39b5e184211bbd04ce1415efd92: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:32:46.143301 19357 raft_consensus.cc:348] T 099453fa543c4196b8898a17e1589412 P d23696d3248b42e2a117be2053d13c10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:46.143399 19357 raft_consensus.cc:374] T 099453fa543c4196b8898a17e1589412 P d23696d3248b42e2a117be2053d13c10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:46.143435 19357 raft_consensus.cc:729] T 099453fa543c4196b8898a17e1589412 P d23696d3248b42e2a117be2053d13c10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d23696d3248b42e2a117be2053d13c10, State: Initialized, Role: FOLLOWER
I20260926 16:32:46.143560 19357 consensus_queue.cc:260] T 099453fa543c4196b8898a17e1589412 P d23696d3248b42e2a117be2053d13c10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:46.143827 19357 ts_tablet_manager.cc:1428] T 099453fa543c4196b8898a17e1589412 P d23696d3248b42e2a117be2053d13c10: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 16:32:46.144721 19354 tablet_bootstrap.cc:492] T 099453fa543c4196b8898a17e1589412 P 3ced2523e6cb40c7a85b38f149ca8e75: Bootstrap starting.
I20260926 16:32:46.145538 19379 ts_tablet_manager.cc:1918] T 987a731df1e14abea8547dae27aaabd6 P d0eec39b5e184211bbd04ce1415efd92: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:32:46.145560 19354 tablet_bootstrap.cc:654] T 099453fa543c4196b8898a17e1589412 P 3ced2523e6cb40c7a85b38f149ca8e75: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:46.145601 19379 log.cc:1196] T 987a731df1e14abea8547dae27aaabd6 P d0eec39b5e184211bbd04ce1415efd92: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-2/wal/wals/987a731df1e14abea8547dae27aaabd6
I20260926 16:32:46.145815 19379 ts_tablet_manager.cc:1939] T 987a731df1e14abea8547dae27aaabd6 P d0eec39b5e184211bbd04ce1415efd92: Deleting consensus metadata
I20260926 16:32:46.146070 19355 tablet_bootstrap.cc:492] T 099453fa543c4196b8898a17e1589412 P d0eec39b5e184211bbd04ce1415efd92: Bootstrap starting.
I20260926 16:32:46.146927 19355 tablet_bootstrap.cc:654] T 099453fa543c4196b8898a17e1589412 P d0eec39b5e184211bbd04ce1415efd92: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:46.146942 18829 catalog_manager.cc:4841] TS d0eec39b5e184211bbd04ce1415efd92 (127.11.8.195:42027): tablet 987a731df1e14abea8547dae27aaabd6 (table test-table [id=14ca7e3c49e94d4dad46cec6066cf473]) successfully deleted
I20260926 16:32:46.148223 19354 tablet_bootstrap.cc:492] T 099453fa543c4196b8898a17e1589412 P 3ced2523e6cb40c7a85b38f149ca8e75: No bootstrap required, opened a new log
I20260926 16:32:46.148310 19354 ts_tablet_manager.cc:1397] T 099453fa543c4196b8898a17e1589412 P 3ced2523e6cb40c7a85b38f149ca8e75: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260926 16:32:46.148746 19354 raft_consensus.cc:348] T 099453fa543c4196b8898a17e1589412 P 3ced2523e6cb40c7a85b38f149ca8e75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:46.148834 19354 raft_consensus.cc:374] T 099453fa543c4196b8898a17e1589412 P 3ced2523e6cb40c7a85b38f149ca8e75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:46.148855 19354 raft_consensus.cc:729] T 099453fa543c4196b8898a17e1589412 P 3ced2523e6cb40c7a85b38f149ca8e75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ced2523e6cb40c7a85b38f149ca8e75, State: Initialized, Role: FOLLOWER
I20260926 16:32:46.148995 19354 consensus_queue.cc:260] T 099453fa543c4196b8898a17e1589412 P 3ced2523e6cb40c7a85b38f149ca8e75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:46.149237 19354 ts_tablet_manager.cc:1428] T 099453fa543c4196b8898a17e1589412 P 3ced2523e6cb40c7a85b38f149ca8e75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260926 16:32:46.149726 19355 tablet_bootstrap.cc:492] T 099453fa543c4196b8898a17e1589412 P d0eec39b5e184211bbd04ce1415efd92: No bootstrap required, opened a new log
I20260926 16:32:46.149808 19355 ts_tablet_manager.cc:1397] T 099453fa543c4196b8898a17e1589412 P d0eec39b5e184211bbd04ce1415efd92: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260926 16:32:46.150293 19355 raft_consensus.cc:348] T 099453fa543c4196b8898a17e1589412 P d0eec39b5e184211bbd04ce1415efd92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:46.150403 19355 raft_consensus.cc:374] T 099453fa543c4196b8898a17e1589412 P d0eec39b5e184211bbd04ce1415efd92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:46.150439 19355 raft_consensus.cc:729] T 099453fa543c4196b8898a17e1589412 P d0eec39b5e184211bbd04ce1415efd92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0eec39b5e184211bbd04ce1415efd92, State: Initialized, Role: FOLLOWER
I20260926 16:32:46.150575 19355 consensus_queue.cc:260] T 099453fa543c4196b8898a17e1589412 P d0eec39b5e184211bbd04ce1415efd92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:46.150817 19355 ts_tablet_manager.cc:1428] T 099453fa543c4196b8898a17e1589412 P d0eec39b5e184211bbd04ce1415efd92: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 16:32:46.152333 18828 catalog_manager.cc:4841] TS 3ced2523e6cb40c7a85b38f149ca8e75 (127.11.8.194:45413): tablet 987a731df1e14abea8547dae27aaabd6 (table test-table [id=14ca7e3c49e94d4dad46cec6066cf473]) successfully deleted
W20260926 16:32:46.159026 19047 tablet.cc:2369] T 099453fa543c4196b8898a17e1589412 P d23696d3248b42e2a117be2053d13c10: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:32:46.228535 19374 raft_consensus.cc:482] T 099453fa543c4196b8898a17e1589412 P d23696d3248b42e2a117be2053d13c10 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:32:46.228700 19374 raft_consensus.cc:504] T 099453fa543c4196b8898a17e1589412 P d23696d3248b42e2a117be2053d13c10 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:46.229200 19374 leader_election.cc:288] T 099453fa543c4196b8898a17e1589412 P d23696d3248b42e2a117be2053d13c10 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ced2523e6cb40c7a85b38f149ca8e75 (127.11.8.194:45413), d0eec39b5e184211bbd04ce1415efd92 (127.11.8.195:42027)
I20260926 16:32:46.229578 19140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "099453fa543c4196b8898a17e1589412" candidate_uuid: "d23696d3248b42e2a117be2053d13c10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" is_pre_election: true
I20260926 16:32:46.229671 19280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "099453fa543c4196b8898a17e1589412" candidate_uuid: "d23696d3248b42e2a117be2053d13c10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0eec39b5e184211bbd04ce1415efd92" is_pre_election: true
I20260926 16:32:46.229753 19140 raft_consensus.cc:2454] T 099453fa543c4196b8898a17e1589412 P 3ced2523e6cb40c7a85b38f149ca8e75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d23696d3248b42e2a117be2053d13c10 in term 0.
I20260926 16:32:46.229843 19280 raft_consensus.cc:2454] T 099453fa543c4196b8898a17e1589412 P d0eec39b5e184211bbd04ce1415efd92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d23696d3248b42e2a117be2053d13c10 in term 0.
I20260926 16:32:46.230113 18932 leader_election.cc:302] T 099453fa543c4196b8898a17e1589412 P d23696d3248b42e2a117be2053d13c10 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ced2523e6cb40c7a85b38f149ca8e75, d23696d3248b42e2a117be2053d13c10; no voters:
I20260926 16:32:46.230327 19374 raft_consensus.cc:2781] T 099453fa543c4196b8898a17e1589412 P d23696d3248b42e2a117be2053d13c10 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:32:46.230412 19374 raft_consensus.cc:482] T 099453fa543c4196b8898a17e1589412 P d23696d3248b42e2a117be2053d13c10 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:32:46.230477 19374 raft_consensus.cc:3037] T 099453fa543c4196b8898a17e1589412 P d23696d3248b42e2a117be2053d13c10 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:46.231509 19374 raft_consensus.cc:504] T 099453fa543c4196b8898a17e1589412 P d23696d3248b42e2a117be2053d13c10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:46.231859 19374 leader_election.cc:288] T 099453fa543c4196b8898a17e1589412 P d23696d3248b42e2a117be2053d13c10 [CANDIDATE]: Term 1 election: Requested vote from peers 3ced2523e6cb40c7a85b38f149ca8e75 (127.11.8.194:45413), d0eec39b5e184211bbd04ce1415efd92 (127.11.8.195:42027)
I20260926 16:32:46.232159 19140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "099453fa543c4196b8898a17e1589412" candidate_uuid: "d23696d3248b42e2a117be2053d13c10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ced2523e6cb40c7a85b38f149ca8e75"
I20260926 16:32:46.232210 19280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "099453fa543c4196b8898a17e1589412" candidate_uuid: "d23696d3248b42e2a117be2053d13c10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0eec39b5e184211bbd04ce1415efd92"
I20260926 16:32:46.232283 19140 raft_consensus.cc:3037] T 099453fa543c4196b8898a17e1589412 P 3ced2523e6cb40c7a85b38f149ca8e75 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:46.232345 19280 raft_consensus.cc:3037] T 099453fa543c4196b8898a17e1589412 P d0eec39b5e184211bbd04ce1415efd92 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:46.233196 19280 raft_consensus.cc:2454] T 099453fa543c4196b8898a17e1589412 P d0eec39b5e184211bbd04ce1415efd92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d23696d3248b42e2a117be2053d13c10 in term 1.
I20260926 16:32:46.233457 18933 leader_election.cc:302] T 099453fa543c4196b8898a17e1589412 P d23696d3248b42e2a117be2053d13c10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0eec39b5e184211bbd04ce1415efd92, d23696d3248b42e2a117be2053d13c10; no voters:
I20260926 16:32:46.233536 19140 raft_consensus.cc:2454] T 099453fa543c4196b8898a17e1589412 P 3ced2523e6cb40c7a85b38f149ca8e75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d23696d3248b42e2a117be2053d13c10 in term 1.
I20260926 16:32:46.233637 19374 raft_consensus.cc:2781] T 099453fa543c4196b8898a17e1589412 P d23696d3248b42e2a117be2053d13c10 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:32:46.233733 19374 raft_consensus.cc:686] T 099453fa543c4196b8898a17e1589412 P d23696d3248b42e2a117be2053d13c10 [term 1 LEADER]: Becoming Leader. State: Replica: d23696d3248b42e2a117be2053d13c10, State: Running, Role: LEADER
I20260926 16:32:46.233896 19374 consensus_queue.cc:237] T 099453fa543c4196b8898a17e1589412 P d23696d3248b42e2a117be2053d13c10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:46.235633 18842 catalog_manager.cc:5495] T 099453fa543c4196b8898a17e1589412 P d23696d3248b42e2a117be2053d13c10 reported cstate change: term changed from 0 to 1, leader changed from <none> to d23696d3248b42e2a117be2053d13c10 (127.11.8.193). New cstate: current_term: 1 leader_uuid: "d23696d3248b42e2a117be2053d13c10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } health_report { overall_health: UNKNOWN } } }
Sep 26 16:32:46 dist-test-slave-2zvq krb5kdc[18798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440365, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.254@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM
I20260926 16:32:46.300514 19390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:46.292055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:38267 (local address 127.0.0.1:45958)
0926 16:32:46.292351 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:46.292363 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:46.292472 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:46.292706 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:46.292708 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:46.293124 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:46.293219 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:46.293224 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:46.294014 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:46.294017 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:46.294604 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:46.294612 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:46.295051 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:46.295053 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:46.295116 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:46.295731 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:46.295742 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:46.297358 (+ 1616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:46.299134 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:46.299142 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:46.299147 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:46.299363 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:46.299631 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:46.299633 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:46.299634 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:46.299706 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:46.300029 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:46.300034 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:46.300128 (+ 94us) client_negotiation.cc:769] Sending connection context
0926 16:32:46.300218 (+ 90us) client_negotiation.cc:240] Negotiation successful
0926 16:32:46.300333 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":93,"threads_started":1}
I20260926 16:32:46.300523 18892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:46.292157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45958 (local address 127.11.8.254:38267)
0926 16:32:46.292338 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:46.292343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:46.292447 (+ 104us) server_negotiation.cc:407] Connection header received
0926 16:32:46.292560 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:46.292563 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:46.292604 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:46.292666 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:46.293358 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:46.293880 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:46.294727 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:46.294941 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:46.295023 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:46.297485 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:46.297488 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:46.297491 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:46.297513 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:46.299042 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:46.299460 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:46.299463 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:46.299464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:46.299551 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:46.299783 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:46.299785 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:46.299786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:46.299944 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:46.300023 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:46.300272 (+ 249us) server_negotiation.cc:299] Negotiation successful
0926 16:32:46.300385 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":104}
I20260926 16:32:46.304164 18842 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:45958:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 16:32:46.304915 18842 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:32:46.309680 18980 tablet_service.cc:1467] Processing CreateTablet for tablet 78751bbd9fed49069f3cc9f49d37872f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8e92aa53235c4755a67b32d412a7a2de]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 16:32:46.309777 19120 tablet_service.cc:1467] Processing CreateTablet for tablet 78751bbd9fed49069f3cc9f49d37872f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8e92aa53235c4755a67b32d412a7a2de]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 16:32:46.309880 19260 tablet_service.cc:1467] Processing CreateTablet for tablet 78751bbd9fed49069f3cc9f49d37872f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8e92aa53235c4755a67b32d412a7a2de]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 16:32:46.310091 18980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78751bbd9fed49069f3cc9f49d37872f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:46.310163 19120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78751bbd9fed49069f3cc9f49d37872f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:46.310199 19260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78751bbd9fed49069f3cc9f49d37872f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:46.312830 19355 tablet_bootstrap.cc:492] T 78751bbd9fed49069f3cc9f49d37872f P d0eec39b5e184211bbd04ce1415efd92: Bootstrap starting.
I20260926 16:32:46.312839 19357 tablet_bootstrap.cc:492] T 78751bbd9fed49069f3cc9f49d37872f P d23696d3248b42e2a117be2053d13c10: Bootstrap starting.
I20260926 16:32:46.312876 19354 tablet_bootstrap.cc:492] T 78751bbd9fed49069f3cc9f49d37872f P 3ced2523e6cb40c7a85b38f149ca8e75: Bootstrap starting.
I20260926 16:32:46.313613 19355 tablet_bootstrap.cc:654] T 78751bbd9fed49069f3cc9f49d37872f P d0eec39b5e184211bbd04ce1415efd92: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:46.313669 19357 tablet_bootstrap.cc:654] T 78751bbd9fed49069f3cc9f49d37872f P d23696d3248b42e2a117be2053d13c10: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:46.313704 19354 tablet_bootstrap.cc:654] T 78751bbd9fed49069f3cc9f49d37872f P 3ced2523e6cb40c7a85b38f149ca8e75: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:46.314733 19355 tablet_bootstrap.cc:492] T 78751bbd9fed49069f3cc9f49d37872f P d0eec39b5e184211bbd04ce1415efd92: No bootstrap required, opened a new log
I20260926 16:32:46.314818 19355 ts_tablet_manager.cc:1397] T 78751bbd9fed49069f3cc9f49d37872f P d0eec39b5e184211bbd04ce1415efd92: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260926 16:32:46.314920 19354 tablet_bootstrap.cc:492] T 78751bbd9fed49069f3cc9f49d37872f P 3ced2523e6cb40c7a85b38f149ca8e75: No bootstrap required, opened a new log
I20260926 16:32:46.314997 19354 ts_tablet_manager.cc:1397] T 78751bbd9fed49069f3cc9f49d37872f P 3ced2523e6cb40c7a85b38f149ca8e75: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 16:32:46.315145 19357 tablet_bootstrap.cc:492] T 78751bbd9fed49069f3cc9f49d37872f P d23696d3248b42e2a117be2053d13c10: No bootstrap required, opened a new log
I20260926 16:32:46.315215 19357 ts_tablet_manager.cc:1397] T 78751bbd9fed49069f3cc9f49d37872f P d23696d3248b42e2a117be2053d13c10: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 16:32:46.315268 19355 raft_consensus.cc:348] T 78751bbd9fed49069f3cc9f49d37872f P d0eec39b5e184211bbd04ce1415efd92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:46.315369 19355 raft_consensus.cc:374] T 78751bbd9fed49069f3cc9f49d37872f P d0eec39b5e184211bbd04ce1415efd92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:46.315398 19355 raft_consensus.cc:729] T 78751bbd9fed49069f3cc9f49d37872f P d0eec39b5e184211bbd04ce1415efd92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0eec39b5e184211bbd04ce1415efd92, State: Initialized, Role: FOLLOWER
I20260926 16:32:46.315425 19354 raft_consensus.cc:348] T 78751bbd9fed49069f3cc9f49d37872f P 3ced2523e6cb40c7a85b38f149ca8e75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:46.315505 19354 raft_consensus.cc:374] T 78751bbd9fed49069f3cc9f49d37872f P 3ced2523e6cb40c7a85b38f149ca8e75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:46.315526 19354 raft_consensus.cc:729] T 78751bbd9fed49069f3cc9f49d37872f P 3ced2523e6cb40c7a85b38f149ca8e75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ced2523e6cb40c7a85b38f149ca8e75, State: Initialized, Role: FOLLOWER
I20260926 16:32:46.315537 19355 consensus_queue.cc:260] T 78751bbd9fed49069f3cc9f49d37872f P d0eec39b5e184211bbd04ce1415efd92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:46.315644 19357 raft_consensus.cc:348] T 78751bbd9fed49069f3cc9f49d37872f P d23696d3248b42e2a117be2053d13c10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:46.315665 19354 consensus_queue.cc:260] T 78751bbd9fed49069f3cc9f49d37872f P 3ced2523e6cb40c7a85b38f149ca8e75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:46.315732 19357 raft_consensus.cc:374] T 78751bbd9fed49069f3cc9f49d37872f P d23696d3248b42e2a117be2053d13c10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:46.315762 19357 raft_consensus.cc:729] T 78751bbd9fed49069f3cc9f49d37872f P d23696d3248b42e2a117be2053d13c10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d23696d3248b42e2a117be2053d13c10, State: Initialized, Role: FOLLOWER
I20260926 16:32:46.315814 19355 ts_tablet_manager.cc:1428] T 78751bbd9fed49069f3cc9f49d37872f P d0eec39b5e184211bbd04ce1415efd92: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260926 16:32:46.315950 19354 ts_tablet_manager.cc:1428] T 78751bbd9fed49069f3cc9f49d37872f P 3ced2523e6cb40c7a85b38f149ca8e75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260926 16:32:46.315896 19360 tablet_replica.cc:439] T 78751bbd9fed49069f3cc9f49d37872f P 3ced2523e6cb40c7a85b38f149ca8e75: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } } }
I20260926 16:32:46.315887 19357 consensus_queue.cc:260] T 78751bbd9fed49069f3cc9f49d37872f P d23696d3248b42e2a117be2053d13c10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:46.316015 19360 tablet_replica.cc:442] T 78751bbd9fed49069f3cc9f49d37872f P 3ced2523e6cb40c7a85b38f149ca8e75: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 16:32:46.316159 19357 ts_tablet_manager.cc:1428] T 78751bbd9fed49069f3cc9f49d37872f P d23696d3248b42e2a117be2053d13c10: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 16:32:46.316375 19361 tablet_replica.cc:439] T 78751bbd9fed49069f3cc9f49d37872f P d0eec39b5e184211bbd04ce1415efd92: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } } }
I20260926 16:32:46.316519 19361 tablet_replica.cc:442] T 78751bbd9fed49069f3cc9f49d37872f P d0eec39b5e184211bbd04ce1415efd92: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 16:32:46.316802 19374 tablet_replica.cc:439] T 78751bbd9fed49069f3cc9f49d37872f P d23696d3248b42e2a117be2053d13c10: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } } }
I20260926 16:32:46.316934 19374 tablet_replica.cc:442] T 78751bbd9fed49069f3cc9f49d37872f P d23696d3248b42e2a117be2053d13c10: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 16:32:46.339634 19374 raft_consensus.cc:482] T 78751bbd9fed49069f3cc9f49d37872f P d23696d3248b42e2a117be2053d13c10 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:32:46.339768 19374 raft_consensus.cc:504] T 78751bbd9fed49069f3cc9f49d37872f P d23696d3248b42e2a117be2053d13c10 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:46.340219 19374 leader_election.cc:288] T 78751bbd9fed49069f3cc9f49d37872f P d23696d3248b42e2a117be2053d13c10 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ced2523e6cb40c7a85b38f149ca8e75 (127.11.8.194:45413), d0eec39b5e184211bbd04ce1415efd92 (127.11.8.195:42027)
I20260926 16:32:46.340657 19140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78751bbd9fed49069f3cc9f49d37872f" candidate_uuid: "d23696d3248b42e2a117be2053d13c10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" is_pre_election: true
I20260926 16:32:46.340657 19280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78751bbd9fed49069f3cc9f49d37872f" candidate_uuid: "d23696d3248b42e2a117be2053d13c10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0eec39b5e184211bbd04ce1415efd92" is_pre_election: true
I20260926 16:32:46.340807 19280 raft_consensus.cc:2454] T 78751bbd9fed49069f3cc9f49d37872f P d0eec39b5e184211bbd04ce1415efd92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d23696d3248b42e2a117be2053d13c10 in term 0.
I20260926 16:32:46.340807 19140 raft_consensus.cc:2454] T 78751bbd9fed49069f3cc9f49d37872f P 3ced2523e6cb40c7a85b38f149ca8e75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d23696d3248b42e2a117be2053d13c10 in term 0.
I20260926 16:32:46.341092 18932 leader_election.cc:302] T 78751bbd9fed49069f3cc9f49d37872f P d23696d3248b42e2a117be2053d13c10 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ced2523e6cb40c7a85b38f149ca8e75, d23696d3248b42e2a117be2053d13c10; no voters:
I20260926 16:32:46.341274 19374 raft_consensus.cc:2781] T 78751bbd9fed49069f3cc9f49d37872f P d23696d3248b42e2a117be2053d13c10 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:32:46.341337 19374 raft_consensus.cc:482] T 78751bbd9fed49069f3cc9f49d37872f P d23696d3248b42e2a117be2053d13c10 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:32:46.341385 19374 raft_consensus.cc:3037] T 78751bbd9fed49069f3cc9f49d37872f P d23696d3248b42e2a117be2053d13c10 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:46.342373 19374 raft_consensus.cc:504] T 78751bbd9fed49069f3cc9f49d37872f P d23696d3248b42e2a117be2053d13c10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:46.342697 19374 leader_election.cc:288] T 78751bbd9fed49069f3cc9f49d37872f P d23696d3248b42e2a117be2053d13c10 [CANDIDATE]: Term 1 election: Requested vote from peers 3ced2523e6cb40c7a85b38f149ca8e75 (127.11.8.194:45413), d0eec39b5e184211bbd04ce1415efd92 (127.11.8.195:42027)
I20260926 16:32:46.343035 19140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78751bbd9fed49069f3cc9f49d37872f" candidate_uuid: "d23696d3248b42e2a117be2053d13c10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ced2523e6cb40c7a85b38f149ca8e75"
I20260926 16:32:46.343077 19280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78751bbd9fed49069f3cc9f49d37872f" candidate_uuid: "d23696d3248b42e2a117be2053d13c10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0eec39b5e184211bbd04ce1415efd92"
I20260926 16:32:46.343166 19140 raft_consensus.cc:3037] T 78751bbd9fed49069f3cc9f49d37872f P 3ced2523e6cb40c7a85b38f149ca8e75 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:46.343184 19280 raft_consensus.cc:3037] T 78751bbd9fed49069f3cc9f49d37872f P d0eec39b5e184211bbd04ce1415efd92 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:46.344218 19140 raft_consensus.cc:2454] T 78751bbd9fed49069f3cc9f49d37872f P 3ced2523e6cb40c7a85b38f149ca8e75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d23696d3248b42e2a117be2053d13c10 in term 1.
I20260926 16:32:46.344218 19280 raft_consensus.cc:2454] T 78751bbd9fed49069f3cc9f49d37872f P d0eec39b5e184211bbd04ce1415efd92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d23696d3248b42e2a117be2053d13c10 in term 1.
I20260926 16:32:46.344501 18933 leader_election.cc:302] T 78751bbd9fed49069f3cc9f49d37872f P d23696d3248b42e2a117be2053d13c10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0eec39b5e184211bbd04ce1415efd92, d23696d3248b42e2a117be2053d13c10; no voters:
I20260926 16:32:46.344714 19374 raft_consensus.cc:2781] T 78751bbd9fed49069f3cc9f49d37872f P d23696d3248b42e2a117be2053d13c10 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:32:46.344810 19374 raft_consensus.cc:686] T 78751bbd9fed49069f3cc9f49d37872f P d23696d3248b42e2a117be2053d13c10 [term 1 LEADER]: Becoming Leader. State: Replica: d23696d3248b42e2a117be2053d13c10, State: Running, Role: LEADER
I20260926 16:32:46.344938 19374 consensus_queue.cc:237] T 78751bbd9fed49069f3cc9f49d37872f P d23696d3248b42e2a117be2053d13c10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:46.345654 19365 tablet_replica.cc:439] T 78751bbd9fed49069f3cc9f49d37872f P d23696d3248b42e2a117be2053d13c10: TxnStatusTablet state changed. Reason: New leader d23696d3248b42e2a117be2053d13c10. Latest consensus state: current_term: 1 leader_uuid: "d23696d3248b42e2a117be2053d13c10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } } }
I20260926 16:32:46.345759 19365 tablet_replica.cc:442] T 78751bbd9fed49069f3cc9f49d37872f P d23696d3248b42e2a117be2053d13c10: This TxnStatusTablet replica's current role is: LEADER
I20260926 16:32:46.346135 19395 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 16:32:46.346350 19395 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 16:32:46.346781 18843 catalog_manager.cc:5495] T 78751bbd9fed49069f3cc9f49d37872f P d23696d3248b42e2a117be2053d13c10 reported cstate change: term changed from 0 to 1, leader changed from <none> to d23696d3248b42e2a117be2053d13c10 (127.11.8.193). New cstate: current_term: 1 leader_uuid: "d23696d3248b42e2a117be2053d13c10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } health_report { overall_health: UNKNOWN } } }
I20260926 16:32:46.366214 19350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:46.362927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54352 (local address 127.11.8.193:42877)
0926 16:32:46.363059 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:46.363064 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:46.363078 (+ 14us) server_negotiation.cc:407] Connection header received
0926 16:32:46.363288 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:46.363296 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:46.363342 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:46.363424 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:32:46.363686 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:46.364242 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:46.365105 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:46.365335 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:46.365447 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:46.365738 (+ 291us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:32:46.365817 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:32:46.365905 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:46.366011 (+ 106us) server_negotiation.cc:299] Negotiation successful
0926 16:32:46.366065 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 16:32:46.366214 19390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:46.362837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:42877 (local address 127.0.0.1:54352)
0926 16:32:46.363005 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:46.363021 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:46.363130 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:46.363446 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:46.363448 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:46.363458 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 16:32:46.363581 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:46.363587 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:46.364349 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:46.364354 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:46.364980 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:46.364988 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:46.365445 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:46.365449 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:46.365548 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:46.365600 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 16:32:46.365906 (+ 306us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 16:32:46.365924 (+ 18us) client_negotiation.cc:769] Sending connection context
0926 16:32:46.365988 (+ 64us) client_negotiation.cc:240] Negotiation successful
0926 16:32:46.366078 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260926 16:32:46.368711 19140 raft_consensus.cc:1261] T 78751bbd9fed49069f3cc9f49d37872f P 3ced2523e6cb40c7a85b38f149ca8e75 [term 1 FOLLOWER]: Refusing update from remote peer d23696d3248b42e2a117be2053d13c10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:32:46.368777 19280 raft_consensus.cc:1261] T 78751bbd9fed49069f3cc9f49d37872f P d0eec39b5e184211bbd04ce1415efd92 [term 1 FOLLOWER]: Refusing update from remote peer d23696d3248b42e2a117be2053d13c10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:32:46.369128 19365 consensus_queue.cc:1035] T 78751bbd9fed49069f3cc9f49d37872f P d23696d3248b42e2a117be2053d13c10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:32:46.369267 19374 consensus_queue.cc:1035] T 78751bbd9fed49069f3cc9f49d37872f P d23696d3248b42e2a117be2053d13c10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:32:46.370612 19361 tablet_replica.cc:439] T 78751bbd9fed49069f3cc9f49d37872f P d0eec39b5e184211bbd04ce1415efd92: TxnStatusTablet state changed. Reason: New leader d23696d3248b42e2a117be2053d13c10. Latest consensus state: current_term: 1 leader_uuid: "d23696d3248b42e2a117be2053d13c10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } } }
I20260926 16:32:46.370738 19361 tablet_replica.cc:442] T 78751bbd9fed49069f3cc9f49d37872f P d0eec39b5e184211bbd04ce1415efd92: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 16:32:46.371984 19360 tablet_replica.cc:439] T 78751bbd9fed49069f3cc9f49d37872f P 3ced2523e6cb40c7a85b38f149ca8e75: TxnStatusTablet state changed. Reason: New leader d23696d3248b42e2a117be2053d13c10. Latest consensus state: current_term: 1 leader_uuid: "d23696d3248b42e2a117be2053d13c10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } } }
I20260926 16:32:46.372116 19360 tablet_replica.cc:442] T 78751bbd9fed49069f3cc9f49d37872f P 3ced2523e6cb40c7a85b38f149ca8e75: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 16:32:46.373241 19365 tablet_replica.cc:439] T 78751bbd9fed49069f3cc9f49d37872f P d23696d3248b42e2a117be2053d13c10: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d23696d3248b42e2a117be2053d13c10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } } }
I20260926 16:32:46.373377 19365 tablet_replica.cc:442] T 78751bbd9fed49069f3cc9f49d37872f P d23696d3248b42e2a117be2053d13c10: This TxnStatusTablet replica's current role is: LEADER
I20260926 16:32:46.373485 19365 tablet_replica.cc:439] T 78751bbd9fed49069f3cc9f49d37872f P d23696d3248b42e2a117be2053d13c10: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d23696d3248b42e2a117be2053d13c10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } } }
I20260926 16:32:46.373550 19365 tablet_replica.cc:442] T 78751bbd9fed49069f3cc9f49d37872f P d23696d3248b42e2a117be2053d13c10: This TxnStatusTablet replica's current role is: LEADER
I20260926 16:32:46.375533 19361 tablet_replica.cc:439] T 78751bbd9fed49069f3cc9f49d37872f P d0eec39b5e184211bbd04ce1415efd92: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d23696d3248b42e2a117be2053d13c10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } } }
I20260926 16:32:46.375643 19361 tablet_replica.cc:442] T 78751bbd9fed49069f3cc9f49d37872f P d0eec39b5e184211bbd04ce1415efd92: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 16:32:46.376919 19360 tablet_replica.cc:439] T 78751bbd9fed49069f3cc9f49d37872f P 3ced2523e6cb40c7a85b38f149ca8e75: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d23696d3248b42e2a117be2053d13c10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } } }
I20260926 16:32:46.377049 19360 tablet_replica.cc:442] T 78751bbd9fed49069f3cc9f49d37872f P 3ced2523e6cb40c7a85b38f149ca8e75: This TxnStatusTablet replica's current role is: FOLLOWER
W20260926 16:32:46.378412 19187 tablet.cc:2369] T 099453fa543c4196b8898a17e1589412 P 3ced2523e6cb40c7a85b38f149ca8e75: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:32:46.388859 19350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:46.385311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:60291 (local address 127.11.8.193:42877)
0926 16:32:46.385441 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:46.385445 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:46.385631 (+ 186us) server_negotiation.cc:407] Connection header received
0926 16:32:46.385762 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:46.385764 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:46.385802 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:46.385861 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:32:46.386127 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:46.386674 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:46.387850 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:46.388061 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:46.388141 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:46.388435 (+ 294us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:32:46.388504 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:32:46.388556 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:46.388675 (+ 119us) server_negotiation.cc:299] Negotiation successful
0926 16:32:46.388728 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":55}
I20260926 16:32:46.389017 19401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:46.385253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:42877 (local address 127.11.8.193:60291)
0926 16:32:46.385561 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:46.385576 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:46.385679 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:46.385891 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:46.385894 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:46.385901 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 16:32:46.386043 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:46.386049 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:46.386771 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:46.386775 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:46.387735 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:46.387746 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:46.388171 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:46.388174 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:46.388254 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:46.388304 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 16:32:46.388587 (+ 283us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 16:32:46.388593 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 16:32:46.388658 (+ 65us) client_negotiation.cc:240] Negotiation successful
0926 16:32:46.388758 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":38,"thread_start_us":123,"threads_started":1}
I20260926 16:32:46.395237 19140 raft_consensus.cc:1261] T 099453fa543c4196b8898a17e1589412 P 3ced2523e6cb40c7a85b38f149ca8e75 [term 1 FOLLOWER]: Refusing update from remote peer d23696d3248b42e2a117be2053d13c10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:32:46.395375 19280 raft_consensus.cc:1261] T 099453fa543c4196b8898a17e1589412 P d0eec39b5e184211bbd04ce1415efd92 [term 1 FOLLOWER]: Refusing update from remote peer d23696d3248b42e2a117be2053d13c10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:32:46.395639 19362 consensus_queue.cc:1035] T 099453fa543c4196b8898a17e1589412 P d23696d3248b42e2a117be2053d13c10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:32:46.395823 19374 consensus_queue.cc:1035] T 099453fa543c4196b8898a17e1589412 P d23696d3248b42e2a117be2053d13c10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:32:46.439649 18841 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45952:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:32:46.439787 18841 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45952:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:32:46.441637 18841 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 507af6a383f0428fab394c8d26d64eaa: Sending DeleteTablet for 3 replicas of tablet 099453fa543c4196b8898a17e1589412
I20260926 16:32:46.442189 19260 tablet_service.cc:1514] Processing DeleteTablet for tablet 099453fa543c4196b8898a17e1589412 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:32:46 UTC) from {username='oryx', principal='oryx/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:46818
I20260926 16:32:46.442214 19120 tablet_service.cc:1514] Processing DeleteTablet for tablet 099453fa543c4196b8898a17e1589412 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:32:46 UTC) from {username='oryx', principal='oryx/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:47832
I20260926 16:32:46.442394 19379 tablet_replica.cc:330] T 099453fa543c4196b8898a17e1589412 P d0eec39b5e184211bbd04ce1415efd92: stopping tablet replica
I20260926 16:32:46.442400 19378 tablet_replica.cc:330] T 099453fa543c4196b8898a17e1589412 P 3ced2523e6cb40c7a85b38f149ca8e75: stopping tablet replica
I20260926 16:32:46.442477 19378 raft_consensus.cc:2229] T 099453fa543c4196b8898a17e1589412 P 3ced2523e6cb40c7a85b38f149ca8e75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:32:46.442493 19379 raft_consensus.cc:2229] T 099453fa543c4196b8898a17e1589412 P d0eec39b5e184211bbd04ce1415efd92 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:32:46.442557 19378 raft_consensus.cc:2258] T 099453fa543c4196b8898a17e1589412 P 3ced2523e6cb40c7a85b38f149ca8e75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:32:46.442586 19379 raft_consensus.cc:2258] T 099453fa543c4196b8898a17e1589412 P d0eec39b5e184211bbd04ce1415efd92 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:32:46.443363 19378 ts_tablet_manager.cc:1905] T 099453fa543c4196b8898a17e1589412 P 3ced2523e6cb40c7a85b38f149ca8e75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:32:46.443393 19379 ts_tablet_manager.cc:1905] T 099453fa543c4196b8898a17e1589412 P d0eec39b5e184211bbd04ce1415efd92: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:32:46.446189 19379 ts_tablet_manager.cc:1918] T 099453fa543c4196b8898a17e1589412 P d0eec39b5e184211bbd04ce1415efd92: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 16:32:46.446295 19379 log.cc:1196] T 099453fa543c4196b8898a17e1589412 P d0eec39b5e184211bbd04ce1415efd92: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-2/wal/wals/099453fa543c4196b8898a17e1589412
I20260926 16:32:46.446566 19379 ts_tablet_manager.cc:1939] T 099453fa543c4196b8898a17e1589412 P d0eec39b5e184211bbd04ce1415efd92: Deleting consensus metadata
I20260926 16:32:46.446926 18980 tablet_service.cc:1514] Processing DeleteTablet for tablet 099453fa543c4196b8898a17e1589412 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:32:46 UTC) from {username='oryx', principal='oryx/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:54322
I20260926 16:32:46.447314 19380 tablet_replica.cc:330] T 099453fa543c4196b8898a17e1589412 P d23696d3248b42e2a117be2053d13c10: stopping tablet replica
I20260926 16:32:46.447481 19380 raft_consensus.cc:2229] T 099453fa543c4196b8898a17e1589412 P d23696d3248b42e2a117be2053d13c10 [term 1 LEADER]: Raft consensus shutting down.
I20260926 16:32:46.447584 18829 catalog_manager.cc:4841] TS d0eec39b5e184211bbd04ce1415efd92 (127.11.8.195:42027): tablet 099453fa543c4196b8898a17e1589412 (table test-table [id=ff0dc02d63fc4939a5179d0b50508b29]) successfully deleted
I20260926 16:32:46.447695 19380 raft_consensus.cc:2258] T 099453fa543c4196b8898a17e1589412 P d23696d3248b42e2a117be2053d13c10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:32:46.447940 19378 ts_tablet_manager.cc:1918] T 099453fa543c4196b8898a17e1589412 P 3ced2523e6cb40c7a85b38f149ca8e75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 16:32:46.448098 19378 log.cc:1196] T 099453fa543c4196b8898a17e1589412 P 3ced2523e6cb40c7a85b38f149ca8e75: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-1/wal/wals/099453fa543c4196b8898a17e1589412
I20260926 16:32:46.448354 19378 ts_tablet_manager.cc:1939] T 099453fa543c4196b8898a17e1589412 P 3ced2523e6cb40c7a85b38f149ca8e75: Deleting consensus metadata
I20260926 16:32:46.448447 19380 ts_tablet_manager.cc:1905] T 099453fa543c4196b8898a17e1589412 P d23696d3248b42e2a117be2053d13c10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:32:46.449096 18828 catalog_manager.cc:4841] TS 3ced2523e6cb40c7a85b38f149ca8e75 (127.11.8.194:45413): tablet 099453fa543c4196b8898a17e1589412 (table test-table [id=ff0dc02d63fc4939a5179d0b50508b29]) successfully deleted
I20260926 16:32:46.451164 19380 ts_tablet_manager.cc:1918] T 099453fa543c4196b8898a17e1589412 P d23696d3248b42e2a117be2053d13c10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 16:32:46.451241 19380 log.cc:1196] T 099453fa543c4196b8898a17e1589412 P d23696d3248b42e2a117be2053d13c10: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-0/wal/wals/099453fa543c4196b8898a17e1589412
I20260926 16:32:46.451486 19380 ts_tablet_manager.cc:1939] T 099453fa543c4196b8898a17e1589412 P d23696d3248b42e2a117be2053d13c10: Deleting consensus metadata
I20260926 16:32:46.452155 18829 catalog_manager.cc:4841] TS d23696d3248b42e2a117be2053d13c10 (127.11.8.193:42877): tablet 099453fa543c4196b8898a17e1589412 (table test-table [id=ff0dc02d63fc4939a5179d0b50508b29]) successfully deleted
Sep 26 16:32:46 dist-test-slave-2zvq krb5kdc[18798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 16:32:46 dist-test-slave-2zvq krb5kdc[18798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:32:46 dist-test-slave-2zvq krb5kdc[18798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
W20260926 16:32:46.475200 18892 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:46.468457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45968 (local address 127.11.8.254:38267)
0926 16:32:46.468598 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:46.468603 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:46.468783 (+ 180us) server_negotiation.cc:407] Connection header received
0926 16:32:46.468949 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:46.468952 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:46.469009 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:46.469103 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:46.469910 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:46.470610 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:46.471580 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:46.471808 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:46.471921 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:46.475077 (+ 3156us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45968: BlockingRecv error: recv got EOF from 127.0.0.1:45968 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 26 16:32:46 dist-test-slave-2zvq krb5kdc[18798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM
I20260926 16:32:46.485987 18892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:46.476927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45980 (local address 127.11.8.254:38267)
0926 16:32:46.477037 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:46.477041 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:46.477057 (+ 16us) server_negotiation.cc:407] Connection header received
0926 16:32:46.477187 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:46.477190 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:46.477238 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:46.477333 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:46.477880 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:46.478678 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:46.479435 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:46.479769 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:46.479873 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:46.482540 (+ 2667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:46.482544 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:46.482546 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:46.482574 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:46.484252 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:46.484684 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:46.484688 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:46.484690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:46.484783 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:46.485079 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:46.485082 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:46.485084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:46.485316 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:46.485473 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:46.485682 (+ 209us) server_negotiation.cc:299] Negotiation successful
0926 16:32:46.485808 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 16:32:46.488996 18841 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45980:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:32:46.489491 18841 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:32:46.494319 19120 tablet_service.cc:1467] Processing CreateTablet for tablet 77dd7edda2674cea95b6e93728792075 (DEFAULT_TABLE table=test-table [id=dffb25765edf447aaaa57609f92e8329]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:46.494654 19120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77dd7edda2674cea95b6e93728792075. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:46.494774 19260 tablet_service.cc:1467] Processing CreateTablet for tablet 77dd7edda2674cea95b6e93728792075 (DEFAULT_TABLE table=test-table [id=dffb25765edf447aaaa57609f92e8329]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:46.495031 19260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77dd7edda2674cea95b6e93728792075. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:46.496894 19354 tablet_bootstrap.cc:492] T 77dd7edda2674cea95b6e93728792075 P 3ced2523e6cb40c7a85b38f149ca8e75: Bootstrap starting.
I20260926 16:32:46.497265 19355 tablet_bootstrap.cc:492] T 77dd7edda2674cea95b6e93728792075 P d0eec39b5e184211bbd04ce1415efd92: Bootstrap starting.
I20260926 16:32:46.497661 19354 tablet_bootstrap.cc:654] T 77dd7edda2674cea95b6e93728792075 P 3ced2523e6cb40c7a85b38f149ca8e75: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:46.498108 19355 tablet_bootstrap.cc:654] T 77dd7edda2674cea95b6e93728792075 P d0eec39b5e184211bbd04ce1415efd92: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:46.494184 18980 tablet_service.cc:1467] Processing CreateTablet for tablet 77dd7edda2674cea95b6e93728792075 (DEFAULT_TABLE table=test-table [id=dffb25765edf447aaaa57609f92e8329]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:46.498517 18980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77dd7edda2674cea95b6e93728792075. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:46.498757 19354 tablet_bootstrap.cc:492] T 77dd7edda2674cea95b6e93728792075 P 3ced2523e6cb40c7a85b38f149ca8e75: No bootstrap required, opened a new log
I20260926 16:32:46.498831 19354 ts_tablet_manager.cc:1397] T 77dd7edda2674cea95b6e93728792075 P 3ced2523e6cb40c7a85b38f149ca8e75: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 16:32:46.499158 19354 raft_consensus.cc:348] T 77dd7edda2674cea95b6e93728792075 P 3ced2523e6cb40c7a85b38f149ca8e75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:46.499236 19354 raft_consensus.cc:374] T 77dd7edda2674cea95b6e93728792075 P 3ced2523e6cb40c7a85b38f149ca8e75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:46.499256 19354 raft_consensus.cc:729] T 77dd7edda2674cea95b6e93728792075 P 3ced2523e6cb40c7a85b38f149ca8e75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ced2523e6cb40c7a85b38f149ca8e75, State: Initialized, Role: FOLLOWER
I20260926 16:32:46.499387 19354 consensus_queue.cc:260] T 77dd7edda2674cea95b6e93728792075 P 3ced2523e6cb40c7a85b38f149ca8e75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:46.499648 19354 ts_tablet_manager.cc:1428] T 77dd7edda2674cea95b6e93728792075 P 3ced2523e6cb40c7a85b38f149ca8e75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 16:32:46.500885 19357 tablet_bootstrap.cc:492] T 77dd7edda2674cea95b6e93728792075 P d23696d3248b42e2a117be2053d13c10: Bootstrap starting.
I20260926 16:32:46.501771 19357 tablet_bootstrap.cc:654] T 77dd7edda2674cea95b6e93728792075 P d23696d3248b42e2a117be2053d13c10: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:46.502985 19357 tablet_bootstrap.cc:492] T 77dd7edda2674cea95b6e93728792075 P d23696d3248b42e2a117be2053d13c10: No bootstrap required, opened a new log
I20260926 16:32:46.503060 19357 ts_tablet_manager.cc:1397] T 77dd7edda2674cea95b6e93728792075 P d23696d3248b42e2a117be2053d13c10: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260926 16:32:46.503415 19357 raft_consensus.cc:348] T 77dd7edda2674cea95b6e93728792075 P d23696d3248b42e2a117be2053d13c10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:46.503508 19357 raft_consensus.cc:374] T 77dd7edda2674cea95b6e93728792075 P d23696d3248b42e2a117be2053d13c10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:46.503535 19357 raft_consensus.cc:729] T 77dd7edda2674cea95b6e93728792075 P d23696d3248b42e2a117be2053d13c10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d23696d3248b42e2a117be2053d13c10, State: Initialized, Role: FOLLOWER
I20260926 16:32:46.503636 19357 consensus_queue.cc:260] T 77dd7edda2674cea95b6e93728792075 P d23696d3248b42e2a117be2053d13c10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:46.503871 19357 ts_tablet_manager.cc:1428] T 77dd7edda2674cea95b6e93728792075 P d23696d3248b42e2a117be2053d13c10: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260926 16:32:46.504683 19355 tablet_bootstrap.cc:492] T 77dd7edda2674cea95b6e93728792075 P d0eec39b5e184211bbd04ce1415efd92: No bootstrap required, opened a new log
I20260926 16:32:46.504755 19355 ts_tablet_manager.cc:1397] T 77dd7edda2674cea95b6e93728792075 P d0eec39b5e184211bbd04ce1415efd92: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20260926 16:32:46.505157 19355 raft_consensus.cc:348] T 77dd7edda2674cea95b6e93728792075 P d0eec39b5e184211bbd04ce1415efd92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:46.505270 19355 raft_consensus.cc:374] T 77dd7edda2674cea95b6e93728792075 P d0eec39b5e184211bbd04ce1415efd92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:46.505312 19355 raft_consensus.cc:729] T 77dd7edda2674cea95b6e93728792075 P d0eec39b5e184211bbd04ce1415efd92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0eec39b5e184211bbd04ce1415efd92, State: Initialized, Role: FOLLOWER
I20260926 16:32:46.505443 19355 consensus_queue.cc:260] T 77dd7edda2674cea95b6e93728792075 P d0eec39b5e184211bbd04ce1415efd92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:46.505728 19355 ts_tablet_manager.cc:1428] T 77dd7edda2674cea95b6e93728792075 P d0eec39b5e184211bbd04ce1415efd92: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260926 16:32:46.684259 19361 raft_consensus.cc:482] T 77dd7edda2674cea95b6e93728792075 P d0eec39b5e184211bbd04ce1415efd92 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:32:46.684414 19361 raft_consensus.cc:504] T 77dd7edda2674cea95b6e93728792075 P d0eec39b5e184211bbd04ce1415efd92 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:46.685559 19361 leader_election.cc:288] T 77dd7edda2674cea95b6e93728792075 P d0eec39b5e184211bbd04ce1415efd92 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d23696d3248b42e2a117be2053d13c10 (127.11.8.193:42877), 3ced2523e6cb40c7a85b38f149ca8e75 (127.11.8.194:45413)
I20260926 16:32:46.690075 19420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:46.685848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:45413 (local address 127.11.8.195:58283)
0926 16:32:46.686332 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:46.686347 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:46.686437 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:46.686731 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:46.686734 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:46.686749 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:46.686842 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:46.686847 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:46.687880 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:46.687884 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:46.688855 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:46.688864 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:46.689639 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:46.689643 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:46.689727 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:46.689753 (+ 26us) client_negotiation.cc:769] Sending connection context
0926 16:32:46.689833 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 16:32:46.689903 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":70,"threads_started":1}
I20260926 16:32:46.690093 19422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:46.686232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:58283 (local address 127.11.8.194:45413)
0926 16:32:46.686504 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:46.686509 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:46.686526 (+ 17us) server_negotiation.cc:407] Connection header received
0926 16:32:46.686574 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:46.686577 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:46.686625 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:46.686708 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:46.687011 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:46.687750 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:46.688983 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:46.689524 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:46.689619 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:46.689661 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:46.689856 (+ 195us) server_negotiation.cc:299] Negotiation successful
0926 16:32:46.689912 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":74,"threads_started":1}
I20260926 16:32:46.690510 19350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:46.685923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:54231 (local address 127.11.8.193:42877)
0926 16:32:46.686104 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:46.686109 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:46.686252 (+ 143us) server_negotiation.cc:407] Connection header received
0926 16:32:46.686510 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:46.686514 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:46.686577 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:46.686626 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:32:46.687562 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:46.688245 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:46.689363 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:46.689878 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:46.689995 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:46.690040 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:46.690305 (+ 265us) server_negotiation.cc:299] Negotiation successful
0926 16:32:46.690380 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260926 16:32:46.690632 19140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77dd7edda2674cea95b6e93728792075" candidate_uuid: "d0eec39b5e184211bbd04ce1415efd92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" is_pre_election: true
I20260926 16:32:46.690783 19140 raft_consensus.cc:2454] T 77dd7edda2674cea95b6e93728792075 P 3ced2523e6cb40c7a85b38f149ca8e75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0eec39b5e184211bbd04ce1415efd92 in term 0.
I20260926 16:32:46.690863 19421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:46.685790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:42877 (local address 127.11.8.195:54231)
0926 16:32:46.686168 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:46.686182 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:46.686361 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:46.686784 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:46.686790 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:46.686816 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:32:46.687408 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:46.687419 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:46.688385 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:46.688388 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:46.689240 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:46.689249 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:46.690021 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:46.690025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:46.690123 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:46.690147 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 16:32:46.690646 (+ 499us) client_negotiation.cc:240] Negotiation successful
0926 16:32:46.690722 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":87,"threads_started":1}
I20260926 16:32:46.691172 19212 leader_election.cc:302] T 77dd7edda2674cea95b6e93728792075 P d0eec39b5e184211bbd04ce1415efd92 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ced2523e6cb40c7a85b38f149ca8e75, d0eec39b5e184211bbd04ce1415efd92; no voters:
I20260926 16:32:46.691435 19361 raft_consensus.cc:2781] T 77dd7edda2674cea95b6e93728792075 P d0eec39b5e184211bbd04ce1415efd92 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:32:46.691403 19000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77dd7edda2674cea95b6e93728792075" candidate_uuid: "d0eec39b5e184211bbd04ce1415efd92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d23696d3248b42e2a117be2053d13c10" is_pre_election: true
I20260926 16:32:46.691498 19361 raft_consensus.cc:482] T 77dd7edda2674cea95b6e93728792075 P d0eec39b5e184211bbd04ce1415efd92 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:32:46.691540 19361 raft_consensus.cc:3037] T 77dd7edda2674cea95b6e93728792075 P d0eec39b5e184211bbd04ce1415efd92 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:46.691651 19000 raft_consensus.cc:2454] T 77dd7edda2674cea95b6e93728792075 P d23696d3248b42e2a117be2053d13c10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0eec39b5e184211bbd04ce1415efd92 in term 0.
I20260926 16:32:46.692790 19361 raft_consensus.cc:504] T 77dd7edda2674cea95b6e93728792075 P d0eec39b5e184211bbd04ce1415efd92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:46.693135 19361 leader_election.cc:288] T 77dd7edda2674cea95b6e93728792075 P d0eec39b5e184211bbd04ce1415efd92 [CANDIDATE]: Term 1 election: Requested vote from peers d23696d3248b42e2a117be2053d13c10 (127.11.8.193:42877), 3ced2523e6cb40c7a85b38f149ca8e75 (127.11.8.194:45413)
I20260926 16:32:46.693444 19000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77dd7edda2674cea95b6e93728792075" candidate_uuid: "d0eec39b5e184211bbd04ce1415efd92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d23696d3248b42e2a117be2053d13c10"
I20260926 16:32:46.693514 19140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77dd7edda2674cea95b6e93728792075" candidate_uuid: "d0eec39b5e184211bbd04ce1415efd92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ced2523e6cb40c7a85b38f149ca8e75"
I20260926 16:32:46.693604 19000 raft_consensus.cc:3037] T 77dd7edda2674cea95b6e93728792075 P d23696d3248b42e2a117be2053d13c10 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:46.693657 19140 raft_consensus.cc:3037] T 77dd7edda2674cea95b6e93728792075 P 3ced2523e6cb40c7a85b38f149ca8e75 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:46.694662 19140 raft_consensus.cc:2454] T 77dd7edda2674cea95b6e93728792075 P 3ced2523e6cb40c7a85b38f149ca8e75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0eec39b5e184211bbd04ce1415efd92 in term 1.
I20260926 16:32:46.694747 19000 raft_consensus.cc:2454] T 77dd7edda2674cea95b6e93728792075 P d23696d3248b42e2a117be2053d13c10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0eec39b5e184211bbd04ce1415efd92 in term 1.
I20260926 16:32:46.694934 19212 leader_election.cc:302] T 77dd7edda2674cea95b6e93728792075 P d0eec39b5e184211bbd04ce1415efd92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ced2523e6cb40c7a85b38f149ca8e75, d0eec39b5e184211bbd04ce1415efd92; no voters:
I20260926 16:32:46.695101 19361 raft_consensus.cc:2781] T 77dd7edda2674cea95b6e93728792075 P d0eec39b5e184211bbd04ce1415efd92 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:32:46.695297 19361 raft_consensus.cc:686] T 77dd7edda2674cea95b6e93728792075 P d0eec39b5e184211bbd04ce1415efd92 [term 1 LEADER]: Becoming Leader. State: Replica: d0eec39b5e184211bbd04ce1415efd92, State: Running, Role: LEADER
I20260926 16:32:46.695544 19361 consensus_queue.cc:237] T 77dd7edda2674cea95b6e93728792075 P d0eec39b5e184211bbd04ce1415efd92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:46.697613 18841 catalog_manager.cc:5495] T 77dd7edda2674cea95b6e93728792075 P d0eec39b5e184211bbd04ce1415efd92 reported cstate change: term changed from 0 to 1, leader changed from <none> to d0eec39b5e184211bbd04ce1415efd92 (127.11.8.195). New cstate: current_term: 1 leader_uuid: "d0eec39b5e184211bbd04ce1415efd92" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } health_report { overall_health: HEALTHY } } }
I20260926 16:32:46.740026 19426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:46.736158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46834 (local address 127.11.8.195:42027)
0926 16:32:46.736509 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:46.736517 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:46.736527 (+ 10us) server_negotiation.cc:407] Connection header received
0926 16:32:46.736600 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:46.736602 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:46.736659 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:46.736788 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:32:46.737040 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:46.737571 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:46.738516 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:46.738754 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:46.738887 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:46.739167 (+ 280us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:32:46.739617 (+ 450us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:32:46.739746 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:46.739806 (+ 60us) server_negotiation.cc:299] Negotiation successful
0926 16:32:46.739858 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":135,"threads_started":1}
I20260926 16:32:46.742460 19140 raft_consensus.cc:1261] T 77dd7edda2674cea95b6e93728792075 P 3ced2523e6cb40c7a85b38f149ca8e75 [term 1 FOLLOWER]: Refusing update from remote peer d0eec39b5e184211bbd04ce1415efd92: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:32:46.742717 19000 raft_consensus.cc:1261] T 77dd7edda2674cea95b6e93728792075 P d23696d3248b42e2a117be2053d13c10 [term 1 FOLLOWER]: Refusing update from remote peer d0eec39b5e184211bbd04ce1415efd92: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:32:46.742942 19361 consensus_queue.cc:1035] T 77dd7edda2674cea95b6e93728792075 P d0eec39b5e184211bbd04ce1415efd92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:32:46.743307 19423 consensus_queue.cc:1035] T 77dd7edda2674cea95b6e93728792075 P d0eec39b5e184211bbd04ce1415efd92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:32:46.753391 18843 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45980:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:32:46.753544 18843 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45980:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:32:46.755445 18843 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 507af6a383f0428fab394c8d26d64eaa: Sending DeleteTablet for 3 replicas of tablet 77dd7edda2674cea95b6e93728792075
I20260926 16:32:46.755962 18980 tablet_service.cc:1514] Processing DeleteTablet for tablet 77dd7edda2674cea95b6e93728792075 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:32:46 UTC) from {username='oryx', principal='oryx/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:54322
I20260926 16:32:46.756141 19380 tablet_replica.cc:330] T 77dd7edda2674cea95b6e93728792075 P d23696d3248b42e2a117be2053d13c10: stopping tablet replica
I20260926 16:32:46.756136 19120 tablet_service.cc:1514] Processing DeleteTablet for tablet 77dd7edda2674cea95b6e93728792075 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:32:46 UTC) from {username='oryx', principal='oryx/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:47832
I20260926 16:32:46.756203 19260 tablet_service.cc:1514] Processing DeleteTablet for tablet 77dd7edda2674cea95b6e93728792075 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:32:46 UTC) from {username='oryx', principal='oryx/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:46818
I20260926 16:32:46.756269 19380 raft_consensus.cc:2229] T 77dd7edda2674cea95b6e93728792075 P d23696d3248b42e2a117be2053d13c10 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:32:46.756311 19378 tablet_replica.cc:330] T 77dd7edda2674cea95b6e93728792075 P 3ced2523e6cb40c7a85b38f149ca8e75: stopping tablet replica
I20260926 16:32:46.756357 19379 tablet_replica.cc:330] T 77dd7edda2674cea95b6e93728792075 P d0eec39b5e184211bbd04ce1415efd92: stopping tablet replica
I20260926 16:32:46.756389 19380 raft_consensus.cc:2258] T 77dd7edda2674cea95b6e93728792075 P d23696d3248b42e2a117be2053d13c10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:32:46.756412 19378 raft_consensus.cc:2229] T 77dd7edda2674cea95b6e93728792075 P 3ced2523e6cb40c7a85b38f149ca8e75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:32:46.756464 19379 raft_consensus.cc:2229] T 77dd7edda2674cea95b6e93728792075 P d0eec39b5e184211bbd04ce1415efd92 [term 1 LEADER]: Raft consensus shutting down.
I20260926 16:32:46.756525 19378 raft_consensus.cc:2258] T 77dd7edda2674cea95b6e93728792075 P 3ced2523e6cb40c7a85b38f149ca8e75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:32:46.756811 19379 raft_consensus.cc:2258] T 77dd7edda2674cea95b6e93728792075 P d0eec39b5e184211bbd04ce1415efd92 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:32:46.757308 19378 ts_tablet_manager.cc:1905] T 77dd7edda2674cea95b6e93728792075 P 3ced2523e6cb40c7a85b38f149ca8e75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:32:46.757371 18843 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45980:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 16:32:46.757493 19379 ts_tablet_manager.cc:1905] T 77dd7edda2674cea95b6e93728792075 P d0eec39b5e184211bbd04ce1415efd92: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 16:32:46.757732 18843 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:32:46.758832 19380 ts_tablet_manager.cc:1905] T 77dd7edda2674cea95b6e93728792075 P d23696d3248b42e2a117be2053d13c10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:32:46.759855 19378 ts_tablet_manager.cc:1918] T 77dd7edda2674cea95b6e93728792075 P 3ced2523e6cb40c7a85b38f149ca8e75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:32:46.759925 19378 log.cc:1196] T 77dd7edda2674cea95b6e93728792075 P 3ced2523e6cb40c7a85b38f149ca8e75: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-1/wal/wals/77dd7edda2674cea95b6e93728792075
I20260926 16:32:46.759913 19379 ts_tablet_manager.cc:1918] T 77dd7edda2674cea95b6e93728792075 P d0eec39b5e184211bbd04ce1415efd92: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:32:46.759991 19379 log.cc:1196] T 77dd7edda2674cea95b6e93728792075 P d0eec39b5e184211bbd04ce1415efd92: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-2/wal/wals/77dd7edda2674cea95b6e93728792075
I20260926 16:32:46.760221 19378 ts_tablet_manager.cc:1939] T 77dd7edda2674cea95b6e93728792075 P 3ced2523e6cb40c7a85b38f149ca8e75: Deleting consensus metadata
I20260926 16:32:46.760257 19379 ts_tablet_manager.cc:1939] T 77dd7edda2674cea95b6e93728792075 P d0eec39b5e184211bbd04ce1415efd92: Deleting consensus metadata
I20260926 16:32:46.760862 18829 catalog_manager.cc:4841] TS d0eec39b5e184211bbd04ce1415efd92 (127.11.8.195:42027): tablet 77dd7edda2674cea95b6e93728792075 (table test-table [id=dffb25765edf447aaaa57609f92e8329]) successfully deleted
I20260926 16:32:46.761021 18828 catalog_manager.cc:4841] TS 3ced2523e6cb40c7a85b38f149ca8e75 (127.11.8.194:45413): tablet 77dd7edda2674cea95b6e93728792075 (table test-table [id=dffb25765edf447aaaa57609f92e8329]) successfully deleted
I20260926 16:32:46.761211 19380 ts_tablet_manager.cc:1918] T 77dd7edda2674cea95b6e93728792075 P d23696d3248b42e2a117be2053d13c10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:32:46.761284 19380 log.cc:1196] T 77dd7edda2674cea95b6e93728792075 P d23696d3248b42e2a117be2053d13c10: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-0/wal/wals/77dd7edda2674cea95b6e93728792075
I20260926 16:32:46.761528 19380 ts_tablet_manager.cc:1939] T 77dd7edda2674cea95b6e93728792075 P d23696d3248b42e2a117be2053d13c10: Deleting consensus metadata
I20260926 16:32:46.762486 18829 catalog_manager.cc:4841] TS d23696d3248b42e2a117be2053d13c10 (127.11.8.193:42877): tablet 77dd7edda2674cea95b6e93728792075 (table test-table [id=dffb25765edf447aaaa57609f92e8329]) successfully deleted
I20260926 16:32:46.765157 18980 tablet_service.cc:1467] Processing CreateTablet for tablet ee32ce30e7f64fbc864da69a45574557 (DEFAULT_TABLE table=test-table [id=2817d82354db484db0e5969edeb0a244]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:46.765225 19120 tablet_service.cc:1467] Processing CreateTablet for tablet ee32ce30e7f64fbc864da69a45574557 (DEFAULT_TABLE table=test-table [id=2817d82354db484db0e5969edeb0a244]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:46.765426 18980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee32ce30e7f64fbc864da69a45574557. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:46.765516 19120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee32ce30e7f64fbc864da69a45574557. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:46.767608 19354 tablet_bootstrap.cc:492] T ee32ce30e7f64fbc864da69a45574557 P 3ced2523e6cb40c7a85b38f149ca8e75: Bootstrap starting.
I20260926 16:32:46.767753 19357 tablet_bootstrap.cc:492] T ee32ce30e7f64fbc864da69a45574557 P d23696d3248b42e2a117be2053d13c10: Bootstrap starting.
I20260926 16:32:46.768435 19354 tablet_bootstrap.cc:654] T ee32ce30e7f64fbc864da69a45574557 P 3ced2523e6cb40c7a85b38f149ca8e75: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:46.768546 19357 tablet_bootstrap.cc:654] T ee32ce30e7f64fbc864da69a45574557 P d23696d3248b42e2a117be2053d13c10: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:46.769647 19354 tablet_bootstrap.cc:492] T ee32ce30e7f64fbc864da69a45574557 P 3ced2523e6cb40c7a85b38f149ca8e75: No bootstrap required, opened a new log
I20260926 16:32:46.769747 19354 ts_tablet_manager.cc:1397] T ee32ce30e7f64fbc864da69a45574557 P 3ced2523e6cb40c7a85b38f149ca8e75: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260926 16:32:46.769742 19260 tablet_service.cc:1467] Processing CreateTablet for tablet ee32ce30e7f64fbc864da69a45574557 (DEFAULT_TABLE table=test-table [id=2817d82354db484db0e5969edeb0a244]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:32:46.770020 19260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee32ce30e7f64fbc864da69a45574557. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:46.770125 19354 raft_consensus.cc:348] T ee32ce30e7f64fbc864da69a45574557 P 3ced2523e6cb40c7a85b38f149ca8e75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:46.770205 19354 raft_consensus.cc:374] T ee32ce30e7f64fbc864da69a45574557 P 3ced2523e6cb40c7a85b38f149ca8e75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:46.770224 19354 raft_consensus.cc:729] T ee32ce30e7f64fbc864da69a45574557 P 3ced2523e6cb40c7a85b38f149ca8e75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ced2523e6cb40c7a85b38f149ca8e75, State: Initialized, Role: FOLLOWER
I20260926 16:32:46.770393 19354 consensus_queue.cc:260] T ee32ce30e7f64fbc864da69a45574557 P 3ced2523e6cb40c7a85b38f149ca8e75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:46.770673 19354 ts_tablet_manager.cc:1428] T ee32ce30e7f64fbc864da69a45574557 P 3ced2523e6cb40c7a85b38f149ca8e75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260926 16:32:46.771366 19357 tablet_bootstrap.cc:492] T ee32ce30e7f64fbc864da69a45574557 P d23696d3248b42e2a117be2053d13c10: No bootstrap required, opened a new log
I20260926 16:32:46.771490 19357 ts_tablet_manager.cc:1397] T ee32ce30e7f64fbc864da69a45574557 P d23696d3248b42e2a117be2053d13c10: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260926 16:32:46.771888 19357 raft_consensus.cc:348] T ee32ce30e7f64fbc864da69a45574557 P d23696d3248b42e2a117be2053d13c10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:46.771994 19357 raft_consensus.cc:374] T ee32ce30e7f64fbc864da69a45574557 P d23696d3248b42e2a117be2053d13c10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:46.772032 19357 raft_consensus.cc:729] T ee32ce30e7f64fbc864da69a45574557 P d23696d3248b42e2a117be2053d13c10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d23696d3248b42e2a117be2053d13c10, State: Initialized, Role: FOLLOWER
I20260926 16:32:46.772140 19357 consensus_queue.cc:260] T ee32ce30e7f64fbc864da69a45574557 P d23696d3248b42e2a117be2053d13c10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:46.772409 19357 ts_tablet_manager.cc:1428] T ee32ce30e7f64fbc864da69a45574557 P d23696d3248b42e2a117be2053d13c10: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260926 16:32:46.772631 19355 tablet_bootstrap.cc:492] T ee32ce30e7f64fbc864da69a45574557 P d0eec39b5e184211bbd04ce1415efd92: Bootstrap starting.
I20260926 16:32:46.773474 19355 tablet_bootstrap.cc:654] T ee32ce30e7f64fbc864da69a45574557 P d0eec39b5e184211bbd04ce1415efd92: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:46.774699 19355 tablet_bootstrap.cc:492] T ee32ce30e7f64fbc864da69a45574557 P d0eec39b5e184211bbd04ce1415efd92: No bootstrap required, opened a new log
I20260926 16:32:46.774770 19355 ts_tablet_manager.cc:1397] T ee32ce30e7f64fbc864da69a45574557 P d0eec39b5e184211bbd04ce1415efd92: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260926 16:32:46.775187 19355 raft_consensus.cc:348] T ee32ce30e7f64fbc864da69a45574557 P d0eec39b5e184211bbd04ce1415efd92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:46.775295 19355 raft_consensus.cc:374] T ee32ce30e7f64fbc864da69a45574557 P d0eec39b5e184211bbd04ce1415efd92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:46.775324 19355 raft_consensus.cc:729] T ee32ce30e7f64fbc864da69a45574557 P d0eec39b5e184211bbd04ce1415efd92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0eec39b5e184211bbd04ce1415efd92, State: Initialized, Role: FOLLOWER
I20260926 16:32:46.775425 19355 consensus_queue.cc:260] T ee32ce30e7f64fbc864da69a45574557 P d0eec39b5e184211bbd04ce1415efd92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:46.775642 19355 ts_tablet_manager.cc:1428] T ee32ce30e7f64fbc864da69a45574557 P d0eec39b5e184211bbd04ce1415efd92: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260926 16:32:47.051759 19362 raft_consensus.cc:482] T ee32ce30e7f64fbc864da69a45574557 P d23696d3248b42e2a117be2053d13c10 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:32:47.051908 19362 raft_consensus.cc:504] T ee32ce30e7f64fbc864da69a45574557 P d23696d3248b42e2a117be2053d13c10 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:47.052280 19362 leader_election.cc:288] T ee32ce30e7f64fbc864da69a45574557 P d23696d3248b42e2a117be2053d13c10 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ced2523e6cb40c7a85b38f149ca8e75 (127.11.8.194:45413), d0eec39b5e184211bbd04ce1415efd92 (127.11.8.195:42027)
I20260926 16:32:47.052830 19140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee32ce30e7f64fbc864da69a45574557" candidate_uuid: "d23696d3248b42e2a117be2053d13c10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" is_pre_election: true
I20260926 16:32:47.052836 19279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee32ce30e7f64fbc864da69a45574557" candidate_uuid: "d23696d3248b42e2a117be2053d13c10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0eec39b5e184211bbd04ce1415efd92" is_pre_election: true
I20260926 16:32:47.053025 19140 raft_consensus.cc:2454] T ee32ce30e7f64fbc864da69a45574557 P 3ced2523e6cb40c7a85b38f149ca8e75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d23696d3248b42e2a117be2053d13c10 in term 0.
I20260926 16:32:47.053048 19279 raft_consensus.cc:2454] T ee32ce30e7f64fbc864da69a45574557 P d0eec39b5e184211bbd04ce1415efd92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d23696d3248b42e2a117be2053d13c10 in term 0.
I20260926 16:32:47.053363 18933 leader_election.cc:302] T ee32ce30e7f64fbc864da69a45574557 P d23696d3248b42e2a117be2053d13c10 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0eec39b5e184211bbd04ce1415efd92, d23696d3248b42e2a117be2053d13c10; no voters:
I20260926 16:32:47.053575 19362 raft_consensus.cc:2781] T ee32ce30e7f64fbc864da69a45574557 P d23696d3248b42e2a117be2053d13c10 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:32:47.053642 19362 raft_consensus.cc:482] T ee32ce30e7f64fbc864da69a45574557 P d23696d3248b42e2a117be2053d13c10 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:32:47.053668 19362 raft_consensus.cc:3037] T ee32ce30e7f64fbc864da69a45574557 P d23696d3248b42e2a117be2053d13c10 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:47.054759 19362 raft_consensus.cc:504] T ee32ce30e7f64fbc864da69a45574557 P d23696d3248b42e2a117be2053d13c10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:47.055110 19362 leader_election.cc:288] T ee32ce30e7f64fbc864da69a45574557 P d23696d3248b42e2a117be2053d13c10 [CANDIDATE]: Term 1 election: Requested vote from peers 3ced2523e6cb40c7a85b38f149ca8e75 (127.11.8.194:45413), d0eec39b5e184211bbd04ce1415efd92 (127.11.8.195:42027)
I20260926 16:32:47.055444 19140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee32ce30e7f64fbc864da69a45574557" candidate_uuid: "d23696d3248b42e2a117be2053d13c10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ced2523e6cb40c7a85b38f149ca8e75"
I20260926 16:32:47.055577 19140 raft_consensus.cc:3037] T ee32ce30e7f64fbc864da69a45574557 P 3ced2523e6cb40c7a85b38f149ca8e75 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:47.055531 19279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee32ce30e7f64fbc864da69a45574557" candidate_uuid: "d23696d3248b42e2a117be2053d13c10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0eec39b5e184211bbd04ce1415efd92"
I20260926 16:32:47.055670 19279 raft_consensus.cc:3037] T ee32ce30e7f64fbc864da69a45574557 P d0eec39b5e184211bbd04ce1415efd92 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:47.056718 19140 raft_consensus.cc:2454] T ee32ce30e7f64fbc864da69a45574557 P 3ced2523e6cb40c7a85b38f149ca8e75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d23696d3248b42e2a117be2053d13c10 in term 1.
I20260926 16:32:47.056978 19279 raft_consensus.cc:2454] T ee32ce30e7f64fbc864da69a45574557 P d0eec39b5e184211bbd04ce1415efd92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d23696d3248b42e2a117be2053d13c10 in term 1.
I20260926 16:32:47.056998 18932 leader_election.cc:302] T ee32ce30e7f64fbc864da69a45574557 P d23696d3248b42e2a117be2053d13c10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ced2523e6cb40c7a85b38f149ca8e75, d23696d3248b42e2a117be2053d13c10; no voters:
I20260926 16:32:47.057150 19362 raft_consensus.cc:2781] T ee32ce30e7f64fbc864da69a45574557 P d23696d3248b42e2a117be2053d13c10 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:32:47.057312 19362 raft_consensus.cc:686] T ee32ce30e7f64fbc864da69a45574557 P d23696d3248b42e2a117be2053d13c10 [term 1 LEADER]: Becoming Leader. State: Replica: d23696d3248b42e2a117be2053d13c10, State: Running, Role: LEADER
I20260926 16:32:47.057454 19362 consensus_queue.cc:237] T ee32ce30e7f64fbc864da69a45574557 P d23696d3248b42e2a117be2053d13c10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } }
I20260926 16:32:47.058970 18841 catalog_manager.cc:5495] T ee32ce30e7f64fbc864da69a45574557 P d23696d3248b42e2a117be2053d13c10 reported cstate change: term changed from 0 to 1, leader changed from <none> to d23696d3248b42e2a117be2053d13c10 (127.11.8.193). New cstate: current_term: 1 leader_uuid: "d23696d3248b42e2a117be2053d13c10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d23696d3248b42e2a117be2053d13c10" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 } health_report { overall_health: UNKNOWN } } }
I20260926 16:32:47.140070 19350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:47.137148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54356 (local address 127.11.8.193:42877)
0926 16:32:47.137275 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:47.137279 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:47.137308 (+ 29us) server_negotiation.cc:407] Connection header received
0926 16:32:47.137395 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:47.137397 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:47.137440 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:47.137498 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:32:47.137681 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:47.138191 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:47.139050 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:47.139257 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:47.139331 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:47.139620 (+ 289us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:32:47.139689 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:32:47.139759 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:47.139845 (+ 86us) server_negotiation.cc:299] Negotiation successful
0926 16:32:47.139888 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 16:32:47.146247 19279 raft_consensus.cc:1261] T ee32ce30e7f64fbc864da69a45574557 P d0eec39b5e184211bbd04ce1415efd92 [term 1 FOLLOWER]: Refusing update from remote peer d23696d3248b42e2a117be2053d13c10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:32:47.146395 19140 raft_consensus.cc:1261] T ee32ce30e7f64fbc864da69a45574557 P 3ced2523e6cb40c7a85b38f149ca8e75 [term 1 FOLLOWER]: Refusing update from remote peer d23696d3248b42e2a117be2053d13c10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:32:47.146770 19436 consensus_queue.cc:1035] T ee32ce30e7f64fbc864da69a45574557 P d23696d3248b42e2a117be2053d13c10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ced2523e6cb40c7a85b38f149ca8e75" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:32:47.146932 19362 consensus_queue.cc:1035] T ee32ce30e7f64fbc864da69a45574557 P d23696d3248b42e2a117be2053d13c10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0eec39b5e184211bbd04ce1415efd92" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 42027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:32:47.179100 18841 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45980:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:32:47.179250 18841 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45980:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:32:47.181066 18841 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 507af6a383f0428fab394c8d26d64eaa: Sending DeleteTablet for 3 replicas of tablet ee32ce30e7f64fbc864da69a45574557
I20260926 16:32:47.181542 18980 tablet_service.cc:1514] Processing DeleteTablet for tablet ee32ce30e7f64fbc864da69a45574557 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:32:47 UTC) from {username='oryx', principal='oryx/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:54322
I20260926 16:32:47.181717 19380 tablet_replica.cc:330] T ee32ce30e7f64fbc864da69a45574557 P d23696d3248b42e2a117be2053d13c10: stopping tablet replica
I20260926 16:32:47.181699 19260 tablet_service.cc:1514] Processing DeleteTablet for tablet ee32ce30e7f64fbc864da69a45574557 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:32:47 UTC) from {username='oryx', principal='oryx/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:46818
I20260926 16:32:47.181855 19380 raft_consensus.cc:2229] T ee32ce30e7f64fbc864da69a45574557 P d23696d3248b42e2a117be2053d13c10 [term 1 LEADER]: Raft consensus shutting down.
I20260926 16:32:47.181900 19379 tablet_replica.cc:330] T ee32ce30e7f64fbc864da69a45574557 P d0eec39b5e184211bbd04ce1415efd92: stopping tablet replica
I20260926 16:32:47.182037 19380 raft_consensus.cc:2258] T ee32ce30e7f64fbc864da69a45574557 P d23696d3248b42e2a117be2053d13c10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:32:47.182054 19379 raft_consensus.cc:2229] T ee32ce30e7f64fbc864da69a45574557 P d0eec39b5e184211bbd04ce1415efd92 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:32:47.182160 19379 raft_consensus.cc:2258] T ee32ce30e7f64fbc864da69a45574557 P d0eec39b5e184211bbd04ce1415efd92 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:32:47.182246 19120 tablet_service.cc:1514] Processing DeleteTablet for tablet ee32ce30e7f64fbc864da69a45574557 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:32:47 UTC) from {username='oryx', principal='oryx/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:47832
I20260926 16:32:47.182442 19378 tablet_replica.cc:330] T ee32ce30e7f64fbc864da69a45574557 P 3ced2523e6cb40c7a85b38f149ca8e75: stopping tablet replica
I20260926 16:32:47.182497 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 18915
I20260926 16:32:47.182559 19378 raft_consensus.cc:2229] T ee32ce30e7f64fbc864da69a45574557 P 3ced2523e6cb40c7a85b38f149ca8e75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:32:47.182672 19378 raft_consensus.cc:2258] T ee32ce30e7f64fbc864da69a45574557 P 3ced2523e6cb40c7a85b38f149ca8e75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:32:47.183943 19379 ts_tablet_manager.cc:1905] T ee32ce30e7f64fbc864da69a45574557 P d0eec39b5e184211bbd04ce1415efd92: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:32:47.183957 19378 ts_tablet_manager.cc:1905] T ee32ce30e7f64fbc864da69a45574557 P 3ced2523e6cb40c7a85b38f149ca8e75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:32:47.186415 19378 ts_tablet_manager.cc:1918] T ee32ce30e7f64fbc864da69a45574557 P 3ced2523e6cb40c7a85b38f149ca8e75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 16:32:47.186415 19379 ts_tablet_manager.cc:1918] T ee32ce30e7f64fbc864da69a45574557 P d0eec39b5e184211bbd04ce1415efd92: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 16:32:47.186496 19379 log.cc:1196] T ee32ce30e7f64fbc864da69a45574557 P d0eec39b5e184211bbd04ce1415efd92: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-2/wal/wals/ee32ce30e7f64fbc864da69a45574557
I20260926 16:32:47.186496 19378 log.cc:1196] T ee32ce30e7f64fbc864da69a45574557 P 3ced2523e6cb40c7a85b38f149ca8e75: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790440324823152-11299-0/minicluster-data/ts-1/wal/wals/ee32ce30e7f64fbc864da69a45574557
I20260926 16:32:47.186758 19379 ts_tablet_manager.cc:1939] T ee32ce30e7f64fbc864da69a45574557 P d0eec39b5e184211bbd04ce1415efd92: Deleting consensus metadata
I20260926 16:32:47.186758 19378 ts_tablet_manager.cc:1939] T ee32ce30e7f64fbc864da69a45574557 P 3ced2523e6cb40c7a85b38f149ca8e75: Deleting consensus metadata
I20260926 16:32:47.187518 18829 catalog_manager.cc:4841] TS d0eec39b5e184211bbd04ce1415efd92 (127.11.8.195:42027): tablet ee32ce30e7f64fbc864da69a45574557 (table test-table [id=2817d82354db484db0e5969edeb0a244]) successfully deleted
I20260926 16:32:47.187613 18828 catalog_manager.cc:4841] TS 3ced2523e6cb40c7a85b38f149ca8e75 (127.11.8.194:45413): tablet ee32ce30e7f64fbc864da69a45574557 (table test-table [id=2817d82354db484db0e5969edeb0a244]) successfully deleted
W20260926 16:32:47.192778 18829 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.8.193:42877 (error 108)
I20260926 16:32:47.192905 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 19055
I20260926 16:32:47.194538 19442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:47.193984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:42877 (local address 127.0.0.1:54370)
0926 16:32:47.194310 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:47.194393 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.193:42877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":59,"threads_started":1}
W20260926 16:32:47.194650 18829 catalog_manager.cc:4584] TS d23696d3248b42e2a117be2053d13c10 (127.11.8.193:42877): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ee32ce30e7f64fbc864da69a45574557: Network error: Client connection negotiation failed: client connection to 127.11.8.193:42877: connect: Connection refused (error 111)
I20260926 16:32:47.198904 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 19195
I20260926 16:32:47.205348 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 18811
2026-09-26T16:32:47Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3441 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:32:47 dist-test-slave-2zvq krb5kdc[19445](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:32:47 dist-test-slave-2zvq krb5kdc[19445](info): set up 2 sockets
Sep 26 16:32:47 dist-test-slave-2zvq krb5kdc[19445](info): commencing operation
krb5kdc: starting...
W20260926 16:32:48.248589 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:32:48 dist-test-slave-2zvq krb5kdc[19445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:32:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:32:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
I20260926 16:32:48.368649 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:35243
--webserver_interface=127.11.8.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:35337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:35243
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:48.479966 19458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:48.480230 19458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:48.480271 19458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:48.484368 19458 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:32:48.484423 19458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:48.484442 19458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:48.484458 19458 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:32:48.484474 19458 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:32:48.489670 19458 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:35337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:35243
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:35243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:48.490688 19458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:48.491698 19458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:48.500072 19466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:48.500095 19464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:48.500137 19463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:48.500301 19458 server_base.cc:1034] running on GCE node
I20260926 16:32:48.501152 19458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:48.502449 19458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:48.503667 19458 hybrid_clock.cc:648] HybridClock initialized: now 1790440368503648 us; error 31 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-26T16:32:48Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1309 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:32:48 dist-test-slave-2zvq krb5kdc[19471](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:32:48 dist-test-slave-2zvq krb5kdc[19471](info): set up 2 sockets
Sep 26 16:32:48 dist-test-slave-2zvq krb5kdc[19471](info): commencing operation
krb5kdc: starting...
W20260926 16:32:49.568751 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:32:49 dist-test-slave-2zvq krb5kdc[19471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:32:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:32:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
I20260926 16:32:49.689814 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:36337
--webserver_interface=127.11.8.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:46183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:36337
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:49.795301 19484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:49.795589 19484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:49.795671 19484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:49.799649 19484 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:32:49.799727 19484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:49.799765 19484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:49.799822 19484 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:32:49.799860 19484 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:32:49.805042 19484 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:46183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:36337
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:36337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:49.806039 19484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:49.807039 19484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:49.815143 19490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:49.815157 19492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:49.815191 19489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:49.815444 19484 server_base.cc:1034] running on GCE node
I20260926 16:32:49.816030 19484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:49.817070 19484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:49.818251 19484 hybrid_clock.cc:648] HybridClock initialized: now 1790440369818226 us; error 46 us; skew 500 ppm
Sep 26 16:32:49 dist-test-slave-2zvq krb5kdc[19471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440369, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:49.827080 19484 init.cc:377] Logged in from keytab as kudu/127.11.8.254@KRBTEST.COM (short username kudu)
I20260926 16:32:49.828008 19484 webserver.cc:460] Webserver started at http://127.11.8.254:40355/ using document root <none> and password file <none>
I20260926 16:32:49.828570 19484 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:49.828660 19484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:49.828888 19484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:49.830956 19484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "907e9aebb3ae4dbf81dbed166daf0fd1"
format_stamp: "Formatted at 2026-09-26 16:32:49 on dist-test-slave-2zvq"
server_key: "5a4f737cc91a75b79566043522bd3c80"
server_key_iv: "5223bbb369a91696691057983b4da268"
server_key_version: "encryptionkey@0"
I20260926 16:32:49.831507 19484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "907e9aebb3ae4dbf81dbed166daf0fd1"
format_stamp: "Formatted at 2026-09-26 16:32:49 on dist-test-slave-2zvq"
server_key: "5a4f737cc91a75b79566043522bd3c80"
server_key_iv: "5223bbb369a91696691057983b4da268"
server_key_version: "encryptionkey@0"
I20260926 16:32:49.835145 19484 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260926 16:32:49.837438 19499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:49.838688 19484 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:32:49.838838 19484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "907e9aebb3ae4dbf81dbed166daf0fd1"
format_stamp: "Formatted at 2026-09-26 16:32:49 on dist-test-slave-2zvq"
server_key: "5a4f737cc91a75b79566043522bd3c80"
server_key_iv: "5223bbb369a91696691057983b4da268"
server_key_version: "encryptionkey@0"
I20260926 16:32:49.838945 19484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:49.857869 19484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:49.861207 19484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:49.861413 19484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:49.868563 19484 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:36337
I20260926 16:32:49.868584 19551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:36337 every 8 connection(s)
I20260926 16:32:49.869498 19484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:32:49.872269 19552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:32:49.875373 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 19484
I20260926 16:32:49.875479 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:32:49.875758 11299 external_mini_cluster.cc:1428] Setting key 70655956e3305f9dbf4c2e1f089716aa
I20260926 16:32:49.878162 19552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1: Bootstrap starting.
I20260926 16:32:49.880326 19552 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1: Neither blocks nor log segments found. Creating new log.
I20260926 16:32:49.881053 19552 log.cc:824] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1: Log is configured to *not* fsync() on all Append() calls
I20260926 16:32:49.883028 19552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1: No bootstrap required, opened a new log
Sep 26 16:32:49 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:32:49.886332 19552 raft_consensus.cc:348] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907e9aebb3ae4dbf81dbed166daf0fd1" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 36337 } }
I20260926 16:32:49.886588 19552 raft_consensus.cc:374] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:32:49.886629 19552 raft_consensus.cc:729] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 907e9aebb3ae4dbf81dbed166daf0fd1, State: Initialized, Role: FOLLOWER
I20260926 16:32:49.887069 19552 consensus_queue.cc:260] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907e9aebb3ae4dbf81dbed166daf0fd1" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 36337 } }
I20260926 16:32:49.887200 19552 raft_consensus.cc:388] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:32:49.887284 19552 raft_consensus.cc:482] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:32:49.887399 19552 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:32:49.888228 19552 raft_consensus.cc:504] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907e9aebb3ae4dbf81dbed166daf0fd1" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 36337 } }
I20260926 16:32:49.888559 19552 leader_election.cc:302] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 907e9aebb3ae4dbf81dbed166daf0fd1; no voters:
I20260926 16:32:49.888867 19552 leader_election.cc:288] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:32:49.889227 19558 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:32:49.889454 19558 raft_consensus.cc:686] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1 [term 1 LEADER]: Becoming Leader. State: Replica: 907e9aebb3ae4dbf81dbed166daf0fd1, State: Running, Role: LEADER
I20260926 16:32:49.889793 19558 consensus_queue.cc:237] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907e9aebb3ae4dbf81dbed166daf0fd1" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 36337 } }
I20260926 16:32:49.890174 19552 sys_catalog.cc:564] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:32:49.890525 19555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:49.877121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43326 (local address 127.11.8.254:36337)
0926 16:32:49.877543 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:49.877558 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:49.877593 (+ 35us) server_negotiation.cc:407] Connection header received
0926 16:32:49.878199 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:49.878215 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:49.878627 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:49.878974 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:49.879741 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:49.880769 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:49.881667 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:49.882127 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:49.882315 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:49.884954 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:49.884960 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:49.884973 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:49.884997 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:49.887490 (+ 2493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:49.887903 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:49.887909 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:49.887914 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:49.888033 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:49.888315 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:49.888318 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:49.888320 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:49.888760 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:49.888974 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:49.889214 (+ 240us) server_negotiation.cc:299] Negotiation successful
0926 16:32:49.889638 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":133,"threads_started":1}
I20260926 16:32:49.891009 19558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 907e9aebb3ae4dbf81dbed166daf0fd1. Latest consensus state: current_term: 1 leader_uuid: "907e9aebb3ae4dbf81dbed166daf0fd1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907e9aebb3ae4dbf81dbed166daf0fd1" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 36337 } } }
I20260926 16:32:49.891723 19558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1 [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:49.891605 19557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "907e9aebb3ae4dbf81dbed166daf0fd1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907e9aebb3ae4dbf81dbed166daf0fd1" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 36337 } } }
I20260926 16:32:49.892038 19557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1 [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:49.892673 19568 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:32:49.895293 19568 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:32:49.900380 19568 catalog_manager.cc:1347] Generated new cluster ID: 1df5c168046c4b9187d59faf2b9cac44
I20260926 16:32:49.900440 19568 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:32:49.907593 19568 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:32:49.908416 19568 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:32:49.923349 19568 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1: Generated new TSK 0
I20260926 16:32:49.924117 19568 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.8.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab.
Entry for principal kudu/127.11.8.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab.
I20260926 16:32:49.952904 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.193:0
--local_ip_for_outbound_sockets=127.11.8.193
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:36337
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:46183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:50.067792 19578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:50.068075 19578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:50.068153 19578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:50.072196 19578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:50.072356 19578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:50.072508 19578 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.193
I20260926 16:32:50.077703 19578 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:46183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:36337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19578
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.193
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:50.078679 19578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:50.079651 19578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:50.088934 19584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:50.089058 19586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:50.088986 19578 server_base.cc:1034] running on GCE node
W20260926 16:32:50.088969 19583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:50.089583 19578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:50.090206 19578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:50.091445 19578 hybrid_clock.cc:648] HybridClock initialized: now 1790440370091426 us; error 62 us; skew 500 ppm
Sep 26 16:32:50 dist-test-slave-2zvq krb5kdc[19471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440370, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:50.095531 19578 init.cc:377] Logged in from keytab as kudu/127.11.8.193@KRBTEST.COM (short username kudu)
I20260926 16:32:50.096728 19578 webserver.cc:460] Webserver started at http://127.11.8.193:43517/ using document root <none> and password file <none>
I20260926 16:32:50.097440 19578 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:50.097534 19578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:50.097783 19578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:50.100500 19578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-0/data/instance:
uuid: "023022d38cbe428083f29de071ff8d36"
format_stamp: "Formatted at 2026-09-26 16:32:50 on dist-test-slave-2zvq"
server_key: "d3bf533dede71ddabde88e34b3f5da10"
server_key_iv: "da5963fa4702b2dbc8175aa5c9d02e63"
server_key_version: "encryptionkey@0"
I20260926 16:32:50.101079 19578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance:
uuid: "023022d38cbe428083f29de071ff8d36"
format_stamp: "Formatted at 2026-09-26 16:32:50 on dist-test-slave-2zvq"
server_key: "d3bf533dede71ddabde88e34b3f5da10"
server_key_iv: "da5963fa4702b2dbc8175aa5c9d02e63"
server_key_version: "encryptionkey@0"
I20260926 16:32:50.104914 19578 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260926 16:32:50.107502 19593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:50.108762 19578 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 16:32:50.108911 19578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-0/wal
uuid: "023022d38cbe428083f29de071ff8d36"
format_stamp: "Formatted at 2026-09-26 16:32:50 on dist-test-slave-2zvq"
server_key: "d3bf533dede71ddabde88e34b3f5da10"
server_key_iv: "da5963fa4702b2dbc8175aa5c9d02e63"
server_key_version: "encryptionkey@0"
I20260926 16:32:50.109037 19578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:50.131111 19578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:50.134275 19578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:50.134498 19578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:50.135179 19578 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:32:50.136061 19578 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:50.136111 19578 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:50.136150 19578 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:50.136166 19578 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:50.145676 19578 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.193:44353
I20260926 16:32:50.145749 19706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.193:44353 every 8 connection(s)
I20260926 16:32:50.146667 19578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
I20260926 16:32:50.149791 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 19578
I20260926 16:32:50.149940 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance
I20260926 16:32:50.150256 11299 external_mini_cluster.cc:1428] Setting key f9957917c7cd37f097c2a41e99dff03a
Sep 26 16:32:50 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440370, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.8.194@KRBTEST.COM; defaulting to no policy
I20260926 16:32:50.161219 19555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:50.148496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:40713 (local address 127.11.8.254:36337)
0926 16:32:50.148631 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:50.148635 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:50.149740 (+ 1105us) server_negotiation.cc:407] Connection header received
0926 16:32:50.150938 (+ 1198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:50.150941 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:50.150985 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:50.151090 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:50.152581 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:50.153106 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:50.154306 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:50.154550 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:50.154673 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:50.157482 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:50.157485 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:50.157487 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:50.157510 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:50.159095 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:50.159647 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:50.159650 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:50.159652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:50.159736 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:50.160079 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:50.160082 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:50.160084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:50.160257 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:50.160367 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:50.160926 (+ 559us) server_negotiation.cc:299] Negotiation successful
0926 16:32:50.161047 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.194@KRBTEST.COM" created.
I20260926 16:32:50.162179 19709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:50.148883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.193:40713)
0926 16:32:50.149564 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:50.149623 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:50.150708 (+ 1085us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:50.151248 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:50.151260 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:50.151742 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:50.152426 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:50.152448 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:50.153248 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:50.153252 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:50.154146 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:50.154155 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:50.154708 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:50.154713 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:50.154892 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:50.155550 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:50.155566 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:50.157294 (+ 1728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:50.159221 (+ 1927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:50.159227 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:50.159241 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:50.159553 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:50.159830 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:50.159834 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:50.159836 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:50.159974 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:50.160368 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:50.160375 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:50.160682 (+ 307us) client_negotiation.cc:769] Sending connection context
0926 16:32:50.160883 (+ 201us) client_negotiation.cc:240] Negotiation successful
0926 16:32:50.161196 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":109,"threads_started":1}
I20260926 16:32:50.163590 19707 heartbeater.cc:347] Connected to a master server at 127.11.8.254:36337
I20260926 16:32:50.163892 19707 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:50.164527 19707 heartbeater.cc:511] Master 127.11.8.254:36337 requested a full tablet report, sending...
I20260926 16:32:50.166236 19516 ts_manager.cc:194] Registered new tserver with Master: 023022d38cbe428083f29de071ff8d36 (127.11.8.193:44353)
I20260926 16:32:50.167438 19516 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.193@KRBTEST.COM'} at 127.11.8.193:40713
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab.
Entry for principal kudu/127.11.8.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab.
I20260926 16:32:50.177613 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.194:0
--local_ip_for_outbound_sockets=127.11.8.194
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:36337
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:46183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:50.285681 19712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:50.285914 19712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:50.285954 19712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:50.289758 19712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:50.289801 19712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:50.289861 19712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.194
I20260926 16:32:50.295174 19712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:46183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:36337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.194
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:50.296063 19712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:50.296980 19712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:50.306288 19720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:50.306315 19717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:50.306288 19718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:50.306627 19712 server_base.cc:1034] running on GCE node
I20260926 16:32:50.307046 19712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:50.307667 19712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:50.308857 19712 hybrid_clock.cc:648] HybridClock initialized: now 1790440370308829 us; error 52 us; skew 500 ppm
Sep 26 16:32:50 dist-test-slave-2zvq krb5kdc[19471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440370, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:50.311558 19712 init.cc:377] Logged in from keytab as kudu/127.11.8.194@KRBTEST.COM (short username kudu)
I20260926 16:32:50.312393 19712 webserver.cc:460] Webserver started at http://127.11.8.194:39647/ using document root <none> and password file <none>
I20260926 16:32:50.312956 19712 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:50.313045 19712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:50.313313 19712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:50.315392 19712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-1/data/instance:
uuid: "38d75c19dda94608b69f481e29665409"
format_stamp: "Formatted at 2026-09-26 16:32:50 on dist-test-slave-2zvq"
server_key: "db1bac91c05b5e9723dd562618df4ccc"
server_key_iv: "5506f719e9666e1f1981f8749bf7d566"
server_key_version: "encryptionkey@0"
I20260926 16:32:50.315958 19712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance:
uuid: "38d75c19dda94608b69f481e29665409"
format_stamp: "Formatted at 2026-09-26 16:32:50 on dist-test-slave-2zvq"
server_key: "db1bac91c05b5e9723dd562618df4ccc"
server_key_iv: "5506f719e9666e1f1981f8749bf7d566"
server_key_version: "encryptionkey@0"
I20260926 16:32:50.319555 19712 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 16:32:50.321863 19727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:50.323173 19712 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:32:50.323307 19712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-1/wal
uuid: "38d75c19dda94608b69f481e29665409"
format_stamp: "Formatted at 2026-09-26 16:32:50 on dist-test-slave-2zvq"
server_key: "db1bac91c05b5e9723dd562618df4ccc"
server_key_iv: "5506f719e9666e1f1981f8749bf7d566"
server_key_version: "encryptionkey@0"
I20260926 16:32:50.323419 19712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:50.337525 19712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:50.340353 19712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:50.340541 19712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:50.341092 19712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:32:50.341928 19712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:50.341993 19712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:50.342063 19712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:50.342115 19712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:50.350948 19712 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.194:39839
I20260926 16:32:50.350979 19840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.194:39839 every 8 connection(s)
I20260926 16:32:50.351779 19712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
I20260926 16:32:50.353432 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 19712
I20260926 16:32:50.353552 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance
I20260926 16:32:50.353861 11299 external_mini_cluster.cc:1428] Setting key f13186bbea7174bd09f77c0c32f566e6
Sep 26 16:32:50 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440370, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.8.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.195@KRBTEST.COM" created.
I20260926 16:32:50.366829 19555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:50.353545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:44237 (local address 127.11.8.254:36337)
0926 16:32:50.353660 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:50.353664 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:50.354981 (+ 1317us) server_negotiation.cc:407] Connection header received
0926 16:32:50.355969 (+ 988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:50.355972 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:50.356023 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:50.356101 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:50.357223 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:50.357754 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:50.359577 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:50.359807 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:50.359898 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:50.362824 (+ 2926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:50.362828 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:50.362831 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:50.362861 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:50.364558 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:50.365097 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:50.365100 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:50.365101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:50.365178 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:50.365532 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:50.365535 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:50.365536 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:50.365712 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:50.365820 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:50.366421 (+ 601us) server_negotiation.cc:299] Negotiation successful
0926 16:32:50.366582 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 16:32:50.367552 19843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:50.354041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.194:44237)
0926 16:32:50.354808 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:50.354862 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:50.355737 (+ 875us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:50.356259 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:50.356266 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:50.356627 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:50.357085 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:50.357098 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:50.357889 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:50.357896 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:50.359450 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:50.359465 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:50.359980 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:50.359987 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:50.360223 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:50.360839 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:50.360866 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:50.362578 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:50.364679 (+ 2101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:50.364684 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:50.364698 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:50.364977 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:50.365275 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:50.365278 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:50.365279 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:50.365425 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:50.365878 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:50.365886 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:50.366109 (+ 223us) client_negotiation.cc:769] Sending connection context
0926 16:32:50.366256 (+ 147us) client_negotiation.cc:240] Negotiation successful
0926 16:32:50.366600 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":192,"threads_started":1}
I20260926 16:32:50.368811 19841 heartbeater.cc:347] Connected to a master server at 127.11.8.254:36337
I20260926 16:32:50.369059 19841 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:50.369642 19841 heartbeater.cc:511] Master 127.11.8.254:36337 requested a full tablet report, sending...
I20260926 16:32:50.370796 19516 ts_manager.cc:194] Registered new tserver with Master: 38d75c19dda94608b69f481e29665409 (127.11.8.194:39839)
I20260926 16:32:50.371353 19516 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.194@KRBTEST.COM'} at 127.11.8.194:44237
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab.
Entry for principal kudu/127.11.8.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab.
I20260926 16:32:50.385824 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.195:0
--local_ip_for_outbound_sockets=127.11.8.195
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:36337
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:46183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:50.495157 19846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:50.495443 19846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:50.495523 19846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:50.499575 19846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:50.499653 19846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:50.499737 19846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.195
I20260926 16:32:50.505158 19846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:46183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:36337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.195
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:50.506170 19846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:50.507193 19846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:50.516908 19852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:50.516930 19846 server_base.cc:1034] running on GCE node
W20260926 16:32:50.516894 19851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:50.516887 19854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:50.517419 19846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:50.517959 19846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:50.519140 19846 hybrid_clock.cc:648] HybridClock initialized: now 1790440370519130 us; error 34 us; skew 500 ppm
Sep 26 16:32:50 dist-test-slave-2zvq krb5kdc[19471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440370, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:50.521715 19846 init.cc:377] Logged in from keytab as kudu/127.11.8.195@KRBTEST.COM (short username kudu)
I20260926 16:32:50.522547 19846 webserver.cc:460] Webserver started at http://127.11.8.195:37347/ using document root <none> and password file <none>
I20260926 16:32:50.523037 19846 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:50.523092 19846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:50.523244 19846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:32:50.525107 19846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-2/data/instance:
uuid: "89696be79ad84d8aadc1cbca93f19e5b"
format_stamp: "Formatted at 2026-09-26 16:32:50 on dist-test-slave-2zvq"
server_key: "95f5b3ad7eeaf8a6754c3f836f37a835"
server_key_iv: "9013fdf1439da0090452630e597dcf50"
server_key_version: "encryptionkey@0"
I20260926 16:32:50.525591 19846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance:
uuid: "89696be79ad84d8aadc1cbca93f19e5b"
format_stamp: "Formatted at 2026-09-26 16:32:50 on dist-test-slave-2zvq"
server_key: "95f5b3ad7eeaf8a6754c3f836f37a835"
server_key_iv: "9013fdf1439da0090452630e597dcf50"
server_key_version: "encryptionkey@0"
I20260926 16:32:50.529237 19846 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260926 16:32:50.531554 19861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:50.532663 19846 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 16:32:50.532796 19846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-2/wal
uuid: "89696be79ad84d8aadc1cbca93f19e5b"
format_stamp: "Formatted at 2026-09-26 16:32:50 on dist-test-slave-2zvq"
server_key: "95f5b3ad7eeaf8a6754c3f836f37a835"
server_key_iv: "9013fdf1439da0090452630e597dcf50"
server_key_version: "encryptionkey@0"
I20260926 16:32:50.532903 19846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:50.552291 19846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:50.554973 19846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:50.555160 19846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:50.555697 19846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:32:50.556458 19846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:50.556535 19846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:50.556622 19846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:50.556671 19846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:50.564682 19846 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.195:45009
I20260926 16:32:50.564754 19974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.195:45009 every 8 connection(s)
I20260926 16:32:50.565527 19846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
I20260926 16:32:50.571637 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 19846
I20260926 16:32:50.571774 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance
I20260926 16:32:50.572073 11299 external_mini_cluster.cc:1428] Setting key bfdf998754c0d28c5f6615a9451d821f
Sep 26 16:32:50 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440370, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:32:50.578466 19555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:50.567271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:58859 (local address 127.11.8.254:36337)
0926 16:32:50.567421 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:50.567426 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:50.568460 (+ 1034us) server_negotiation.cc:407] Connection header received
0926 16:32:50.569259 (+ 799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:50.569262 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:50.569306 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:50.569377 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:50.570473 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:50.571009 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:50.572293 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:50.572547 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:50.572634 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:50.575233 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:50.575236 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:50.575238 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:50.575261 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:50.576493 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:50.576950 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:50.576952 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:50.576954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:50.577024 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:50.577328 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:50.577334 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:50.577337 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:50.577534 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:50.577609 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:50.578157 (+ 548us) server_negotiation.cc:299] Negotiation successful
0926 16:32:50.578344 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 16:32:50.579257 19977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:50.567679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.195:58859)
0926 16:32:50.568302 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:50.568340 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:50.569063 (+ 723us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:50.569544 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:50.569550 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:50.569877 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:50.570330 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:50.570342 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:50.571140 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:50.571143 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:50.572170 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:50.572182 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:50.572682 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:50.572688 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:50.572883 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:50.573539 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:50.573553 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:50.575092 (+ 1539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:32:50.576608 (+ 1516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:50.576612 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:50.576623 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:50.576868 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:50.577104 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:32:50.577106 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:32:50.577107 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:32:50.577227 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:32:50.577662 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:32:50.577667 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:32:50.577884 (+ 217us) client_negotiation.cc:769] Sending connection context
0926 16:32:50.578081 (+ 197us) client_negotiation.cc:240] Negotiation successful
0926 16:32:50.578410 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":118,"threads_started":1}
I20260926 16:32:50.580492 19975 heartbeater.cc:347] Connected to a master server at 127.11.8.254:36337
I20260926 16:32:50.580698 19975 heartbeater.cc:464] Registering TS with master...
I20260926 16:32:50.581130 19975 heartbeater.cc:511] Master 127.11.8.254:36337 requested a full tablet report, sending...
I20260926 16:32:50.582080 19516 ts_manager.cc:194] Registered new tserver with Master: 89696be79ad84d8aadc1cbca93f19e5b (127.11.8.195:45009)
I20260926 16:32:50.582681 19516 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.195@KRBTEST.COM'} at 127.11.8.195:58859
I20260926 16:32:50.585985 11299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.11.8.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.8.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.8.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.193.keytab.
Entry for principal oryx/127.11.8.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.193.keytab.
WARNING: no policy specified for oryx/127.11.8.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.8.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.8.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.194.keytab.
Entry for principal oryx/127.11.8.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.194.keytab.
WARNING: no policy specified for oryx/127.11.8.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.8.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.8.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.195.keytab.
Entry for principal oryx/127.11.8.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.195.keytab.
I20260926 16:32:50.660979 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 19578
I20260926 16:32:50.666543 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 19712
I20260926 16:32:50.671815 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 19846
I20260926 16:32:50.676976 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 19484
I20260926 16:32:50.683233 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:36337
--webserver_interface=127.11.8.254
--webserver_port=40355
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:46183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:36337
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:32:50.794975 19984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:50.795246 19984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:50.795320 19984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:50.799367 19984 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:32:50.799446 19984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:50.799486 19984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:50.799541 19984 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:32:50.799579 19984 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:32:50.804630 19984 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:46183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:36337
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:36337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=40355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:50.805625 19984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:50.806664 19984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:50.814899 19989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:50.814883 19990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:50.815001 19984 server_base.cc:1034] running on GCE node
W20260926 16:32:50.814877 19992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:50.815713 19984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:50.816802 19984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:50.817991 19984 hybrid_clock.cc:648] HybridClock initialized: now 1790440370817974 us; error 31 us; skew 500 ppm
Sep 26 16:32:50 dist-test-slave-2zvq krb5kdc[19471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440370, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:50.820668 19984 init.cc:377] Logged in from keytab as kudu/127.11.8.254@KRBTEST.COM (short username kudu)
I20260926 16:32:50.821507 19984 webserver.cc:460] Webserver started at http://127.11.8.254:40355/ using document root <none> and password file <none>
I20260926 16:32:50.822044 19984 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:50.822154 19984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:50.825840 19984 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260926 16:32:50.827973 19999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:50.829226 19984 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 16:32:50.829360 19984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "907e9aebb3ae4dbf81dbed166daf0fd1"
format_stamp: "Formatted at 2026-09-26 16:32:49 on dist-test-slave-2zvq"
server_key: "5a4f737cc91a75b79566043522bd3c80"
server_key_iv: "5223bbb369a91696691057983b4da268"
server_key_version: "encryptionkey@0"
I20260926 16:32:50.829895 19984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:50.841869 19984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:50.844628 19984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:50.844815 19984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:50.851692 19984 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:36337
I20260926 16:32:50.851754 20051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:36337 every 8 connection(s)
I20260926 16:32:50.852718 19984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:32:50.856988 20052 sys_catalog.cc:263] Verifying existing consensus state
I20260926 16:32:50.858695 20052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1: Bootstrap starting.
I20260926 16:32:50.858901 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 19984
I20260926 16:32:50.859352 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.193:44353
--local_ip_for_outbound_sockets=127.11.8.193
--tserver_master_addrs=127.11.8.254:36337
--webserver_port=43517
--webserver_interface=127.11.8.193
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:46183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.8.193
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 16:32:50.876982 20052 log.cc:824] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1: Log is configured to *not* fsync() on all Append() calls
I20260926 16:32:50.884322 20052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 16:32:50.884702 20052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1: Bootstrap complete.
I20260926 16:32:50.889146 20052 raft_consensus.cc:348] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907e9aebb3ae4dbf81dbed166daf0fd1" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 36337 } }
I20260926 16:32:50.889585 20052 raft_consensus.cc:729] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 907e9aebb3ae4dbf81dbed166daf0fd1, State: Initialized, Role: FOLLOWER
I20260926 16:32:50.890079 20052 consensus_queue.cc:260] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907e9aebb3ae4dbf81dbed166daf0fd1" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 36337 } }
I20260926 16:32:50.890228 20052 raft_consensus.cc:388] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:32:50.890350 20052 raft_consensus.cc:482] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:32:50.890463 20052 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1 [term 1 FOLLOWER]: Advancing to term 2
I20260926 16:32:50.892408 20052 raft_consensus.cc:504] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907e9aebb3ae4dbf81dbed166daf0fd1" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 36337 } }
I20260926 16:32:50.892757 20052 leader_election.cc:302] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 907e9aebb3ae4dbf81dbed166daf0fd1; no voters:
I20260926 16:32:50.893108 20052 leader_election.cc:288] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 16:32:50.893203 20056 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 16:32:50.893536 20056 raft_consensus.cc:686] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1 [term 2 LEADER]: Becoming Leader. State: Replica: 907e9aebb3ae4dbf81dbed166daf0fd1, State: Running, Role: LEADER
I20260926 16:32:50.893896 20056 consensus_queue.cc:237] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907e9aebb3ae4dbf81dbed166daf0fd1" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 36337 } }
I20260926 16:32:50.894407 20052 sys_catalog.cc:564] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:32:50.895318 20057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "907e9aebb3ae4dbf81dbed166daf0fd1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907e9aebb3ae4dbf81dbed166daf0fd1" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 36337 } } }
I20260926 16:32:50.895478 20057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1 [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:50.895808 20058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 907e9aebb3ae4dbf81dbed166daf0fd1. Latest consensus state: current_term: 2 leader_uuid: "907e9aebb3ae4dbf81dbed166daf0fd1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907e9aebb3ae4dbf81dbed166daf0fd1" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 36337 } } }
I20260926 16:32:50.895932 20058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1 [sys.catalog]: This master's current role is: LEADER
I20260926 16:32:50.896270 20065 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:32:50.899466 20065 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:32:50.900307 20065 catalog_manager.cc:1259] Loaded cluster ID: 1df5c168046c4b9187d59faf2b9cac44
I20260926 16:32:50.900367 20065 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:32:50.902668 20065 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:32:50.903337 20065 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 907e9aebb3ae4dbf81dbed166daf0fd1: Loaded TSK: 0
I20260926 16:32:50.904778 20065 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 16:32:50.974220 20054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:50.974506 20054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:50.974584 20054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:50.978679 20054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:50.978745 20054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:50.978847 20054 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.193
I20260926 16:32:50.984053 20054 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:46183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.193.keytab
--principal=oryx/127.11.8.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.193:44353
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.193
--webserver_port=43517
--tserver_master_addrs=127.11.8.254:36337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.193
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:50.985047 20054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:50.986029 20054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:50.996588 20079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:50.996658 20081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:50.996583 20078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:50.996945 20054 server_base.cc:1034] running on GCE node
I20260926 16:32:50.997417 20054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:50.998082 20054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:50.999265 20054 hybrid_clock.cc:648] HybridClock initialized: now 1790440370999243 us; error 68 us; skew 500 ppm
Sep 26 16:32:50 dist-test-slave-2zvq krb5kdc[19471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440370, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:51.002255 20054 init.cc:377] Logged in from keytab as oryx/127.11.8.193@KRBTEST.COM (short username oryx)
I20260926 16:32:51.003221 20054 webserver.cc:460] Webserver started at http://127.11.8.193:43517/ using document root <none> and password file <none>
I20260926 16:32:51.003775 20054 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:51.003847 20054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:51.007773 20054 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260926 16:32:51.009830 20088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:51.011286 20054 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 16:32:51.011376 20054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-0/wal
uuid: "023022d38cbe428083f29de071ff8d36"
format_stamp: "Formatted at 2026-09-26 16:32:50 on dist-test-slave-2zvq"
server_key: "d3bf533dede71ddabde88e34b3f5da10"
server_key_iv: "da5963fa4702b2dbc8175aa5c9d02e63"
server_key_version: "encryptionkey@0"
I20260926 16:32:51.011991 20054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:51.049437 20054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:51.052609 20054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:51.052764 20054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:51.053350 20054 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:32:51.054180 20054 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:51.054252 20054 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:51.054340 20054 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:51.054385 20054 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:51.063520 20054 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.193:44353
I20260926 16:32:51.063573 20201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.193:44353 every 8 connection(s)
I20260926 16:32:51.064623 20054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
I20260926 16:32:51.066411 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 20054
I20260926 16:32:51.066846 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.194:39839
--local_ip_for_outbound_sockets=127.11.8.194
--tserver_master_addrs=127.11.8.254:36337
--webserver_port=39647
--webserver_interface=127.11.8.194
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:46183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.8.194
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 16:32:51 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.193@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:32:51 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.193@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
W20260926 16:32:51.079963 20204 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:51.066834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:60855 (local address 127.11.8.254:36337)
0926 16:32:51.067395 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:51.067448 (+ 53us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:51.068341 (+ 893us) server_negotiation.cc:407] Connection header received
0926 16:32:51.069759 (+ 1418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:51.069769 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:51.070059 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:51.070378 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:51.071946 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:51.072899 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:51.074792 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:51.075247 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:51.075363 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:51.079231 (+ 3868us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.193:60855: BlockingRecv error: recv got EOF from 127.11.8.193:60855 (error 108)
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":123,"threads_started":1}
I20260926 16:32:51.079970 20205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:51.066971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.193:60855)
0926 16:32:51.068149 (+ 1178us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:51.068208 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:51.069353 (+ 1145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:51.070492 (+ 1139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:51.070502 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:51.071043 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:51.071763 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:51.071788 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:51.073055 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:51.073062 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:51.074664 (+ 1602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:51.074679 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:51.075417 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:51.075422 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:51.075656 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:51.076297 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:51.076324 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:51.079019 (+ 2695us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":933,"thread_start_us":106,"threads_started":1}
W20260926 16:32:51.080641 20202 heartbeater.cc:644] Failed to heartbeat to 127.11.8.254:36337 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.8.254:36337: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Sep 26 16:32:51 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.193@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:32:51 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.193@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:32:51.086674 20205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:51.081099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.193:40319)
0926 16:32:51.081263 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:51.081279 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:51.081368 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:51.081650 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:51.081652 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:51.081791 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:51.081922 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:51.081927 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:51.082950 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:51.082952 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:51.083540 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:51.083548 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:51.084150 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:51.084154 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:51.084230 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:51.084730 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:51.084742 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:51.086579 (+ 1837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":76}
W20260926 16:32:51.086831 20204 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:51.081306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:40319 (local address 127.11.8.254:36337)
0926 16:32:51.081441 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:51.081445 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:51.081459 (+ 14us) server_negotiation.cc:407] Connection header received
0926 16:32:51.081494 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:51.081497 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:51.081540 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:51.081623 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:51.082035 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:51.082840 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:51.083664 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:51.084024 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:51.084126 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:51.086728 (+ 2602us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.193:40319: BlockingRecv error: recv got EOF from 127.11.8.193:40319 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 26 16:32:51 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.193@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:32:51 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.193@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:32:51.093173 20207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:51.087240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.193:47087)
0926 16:32:51.087559 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:51.087572 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:51.087668 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:51.087926 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:51.087928 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:51.088139 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:51.088254 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:51.088260 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:51.089005 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:51.089008 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:51.089903 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:51.089914 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:51.090434 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:51.090437 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:51.090542 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:51.091039 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:51.091053 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:51.092988 (+ 1935us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":109,"threads_started":1}
W20260926 16:32:51.093223 20204 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:51.087301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:47087 (local address 127.11.8.254:36337)
0926 16:32:51.087436 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:51.087441 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:51.087650 (+ 209us) server_negotiation.cc:407] Connection header received
0926 16:32:51.087778 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:51.087780 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:51.087818 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:51.087891 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:51.088360 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:51.088890 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:51.090029 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:51.090257 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:51.090413 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:51.093112 (+ 2699us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.193:47087: BlockingRecv error: recv got EOF from 127.11.8.193:47087 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260926 16:32:51.093390 20202 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 16:32:51.182708 20206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:51.182977 20206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:51.183049 20206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:51.187083 20206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:51.187161 20206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:51.187247 20206 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.194
I20260926 16:32:51.192633 20206 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:46183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.194.keytab
--principal=oryx/127.11.8.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.194:39839
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.194
--webserver_port=39647
--tserver_master_addrs=127.11.8.254:36337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.194
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:51.193593 20206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:51.194626 20206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:51.204133 20213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:51.204120 20212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:51.204120 20215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:51.204258 20206 server_base.cc:1034] running on GCE node
I20260926 16:32:51.204795 20206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:51.205397 20206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:51.206580 20206 hybrid_clock.cc:648] HybridClock initialized: now 1790440371206567 us; error 29 us; skew 500 ppm
Sep 26 16:32:51 dist-test-slave-2zvq krb5kdc[19471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440371, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:51.209141 20206 init.cc:377] Logged in from keytab as oryx/127.11.8.194@KRBTEST.COM (short username oryx)
I20260926 16:32:51.209986 20206 webserver.cc:460] Webserver started at http://127.11.8.194:39647/ using document root <none> and password file <none>
I20260926 16:32:51.210598 20206 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:51.210709 20206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:51.214663 20206 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260926 16:32:51.216737 20222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:51.218024 20206 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:32:51.218155 20206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-1/wal
uuid: "38d75c19dda94608b69f481e29665409"
format_stamp: "Formatted at 2026-09-26 16:32:50 on dist-test-slave-2zvq"
server_key: "db1bac91c05b5e9723dd562618df4ccc"
server_key_iv: "5506f719e9666e1f1981f8749bf7d566"
server_key_version: "encryptionkey@0"
I20260926 16:32:51.218734 20206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:51.234509 20206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:51.237215 20206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:51.237425 20206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:51.237969 20206 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:32:51.238842 20206 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:51.238912 20206 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:51.238978 20206 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:51.239038 20206 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:51.248032 20206 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.194:39839
I20260926 16:32:51.248054 20335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.194:39839 every 8 connection(s)
I20260926 16:32:51.248936 20206 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
I20260926 16:32:51.253654 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 20206
I20260926 16:32:51.254132 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.195:45009
--local_ip_for_outbound_sockets=127.11.8.195
--tserver_master_addrs=127.11.8.254:36337
--webserver_port=37347
--webserver_interface=127.11.8.195
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:46183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.8.195
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 16:32:51 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.194@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:32:51 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.194@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
W20260926 16:32:51.260456 20204 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:51.250638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:39733 (local address 127.11.8.254:36337)
0926 16:32:51.250769 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:51.250773 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:51.251797 (+ 1024us) server_negotiation.cc:407] Connection header received
0926 16:32:51.252690 (+ 893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:51.252693 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:51.252753 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:51.252864 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:51.254318 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:51.254863 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:51.256216 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:51.256619 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:51.256745 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:51.260282 (+ 3537us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.194:39733: BlockingRecv error: recv got EOF from 127.11.8.194:39733 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 16:32:51.260885 20338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:51.251007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.194:39733)
0926 16:32:51.251652 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:51.251690 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:51.252488 (+ 798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:51.253029 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:51.253036 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:51.253443 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:51.254108 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:51.254127 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:51.254996 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:51.255003 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:51.256078 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:51.256088 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:51.256789 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:51.256792 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:51.256976 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:51.257618 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:51.257633 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:51.260080 (+ 2447us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":103,"threads_started":1}
W20260926 16:32:51.261471 20336 heartbeater.cc:644] Failed to heartbeat to 127.11.8.254:36337 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.8.254:36337: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Sep 26 16:32:51 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.194@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:32:51 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.194@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:32:51.267480 20338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:51.261948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.194:57493)
0926 16:32:51.262059 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:51.262075 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:51.262151 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:51.262462 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:51.262464 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:51.262587 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:51.262673 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:51.262678 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:51.263797 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:51.263800 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:51.264375 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:51.264419 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:51.265099 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:51.265102 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:51.265173 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:51.265662 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:51.265673 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:51.267398 (+ 1725us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":47}
W20260926 16:32:51.267831 20204 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:51.262041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:57493 (local address 127.11.8.254:36337)
0926 16:32:51.262183 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:51.262189 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:51.262205 (+ 16us) server_negotiation.cc:407] Connection header received
0926 16:32:51.262251 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:51.262255 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:51.262331 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:51.262421 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:51.262787 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:51.263669 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:51.264557 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:51.264969 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:51.265077 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:51.267544 (+ 2467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.194:57493: BlockingRecv error: recv got EOF from 127.11.8.194:57493 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 26 16:32:51 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.194@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:32:51 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.194@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:32:51.273244 20338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:51.267947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.194:42675)
0926 16:32:51.268093 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:51.268104 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:51.268157 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:51.268525 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:51.268527 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:51.268645 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:51.268707 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:51.268711 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:51.269678 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:51.269680 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:51.270198 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:51.270205 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:51.270798 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:51.270800 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:51.270894 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:51.271359 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:51.271370 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:51.273135 (+ 1765us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":88}
W20260926 16:32:51.273469 20204 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:51.268065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:42675 (local address 127.11.8.254:36337)
0926 16:32:51.268333 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:51.268337 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:51.268350 (+ 13us) server_negotiation.cc:407] Connection header received
0926 16:32:51.268383 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:51.268386 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:51.268422 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:51.268507 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:51.268804 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:51.269576 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:51.270332 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:51.270675 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:51.270775 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:51.273259 (+ 2484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.194:42675: BlockingRecv error: recv got EOF from 127.11.8.194:42675 (error 108)
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":107,"threads_started":1}
W20260926 16:32:51.273638 20336 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 16:32:51.365936 20339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:32:51.366216 20339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:32:51.366309 20339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:32:51.370281 20339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:32:51.370368 20339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:32:51.370461 20339 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.195
I20260926 16:32:51.375581 20339 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:46183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/krb5kdc/oryx_127.11.8.195.keytab
--principal=oryx/127.11.8.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.195:45009
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.195
--webserver_port=37347
--tserver_master_addrs=127.11.8.254:36337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20339
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.195
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:32:51.376613 20339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:32:51.377630 20339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:32:51.387418 20348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:51.387423 20346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:32:51.387421 20345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:32:51.387692 20339 server_base.cc:1034] running on GCE node
I20260926 16:32:51.388234 20339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:32:51.388854 20339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:32:51.390025 20339 hybrid_clock.cc:648] HybridClock initialized: now 1790440371390010 us; error 26 us; skew 500 ppm
Sep 26 16:32:51 dist-test-slave-2zvq krb5kdc[19471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440371, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:32:51.392711 20339 init.cc:377] Logged in from keytab as oryx/127.11.8.195@KRBTEST.COM (short username oryx)
I20260926 16:32:51.393505 20339 webserver.cc:460] Webserver started at http://127.11.8.195:37347/ using document root <none> and password file <none>
I20260926 16:32:51.394014 20339 fs_manager.cc:359] Metadata directory not provided
I20260926 16:32:51.394083 20339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:32:51.398258 20339 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:32:51.400592 20355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:51.401914 20339 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 16:32:51.402022 20339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-2/wal
uuid: "89696be79ad84d8aadc1cbca93f19e5b"
format_stamp: "Formatted at 2026-09-26 16:32:50 on dist-test-slave-2zvq"
server_key: "95f5b3ad7eeaf8a6754c3f836f37a835"
server_key_iv: "9013fdf1439da0090452630e597dcf50"
server_key_version: "encryptionkey@0"
I20260926 16:32:51.402637 20339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:32:51.417577 20339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:32:51.420652 20339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:32:51.420797 20339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:32:51.421350 20339 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:32:51.422238 20339 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:32:51.422315 20339 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:51.422385 20339 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:32:51.422420 20339 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:32:51.430925 20339 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.195:45009
I20260926 16:32:51.431025 20468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.195:45009 every 8 connection(s)
I20260926 16:32:51.431902 20339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
I20260926 16:32:51.440757 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 20339
Sep 26 16:32:51 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.195@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:32:51 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.195@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
W20260926 16:32:51.442935 20204 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:51.433692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:47229 (local address 127.11.8.254:36337)
0926 16:32:51.433826 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:51.433829 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:51.434747 (+ 918us) server_negotiation.cc:407] Connection header received
0926 16:32:51.436001 (+ 1254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:51.436004 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:51.436064 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:51.436165 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:51.437270 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:51.437786 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:51.438999 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:51.439221 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:51.439336 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:51.442839 (+ 3503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.195:47229: BlockingRecv error: recv got EOF from 127.11.8.195:47229 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 16:32:51.443648 20471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:51.434130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.195:47229)
0926 16:32:51.434605 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:51.434644 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:51.435791 (+ 1147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:51.436349 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:51.436357 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:51.436685 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:51.437128 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:51.437141 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:51.437912 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:51.437915 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:51.438873 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:51.438882 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:51.439386 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:51.439391 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:51.439551 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:51.440122 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:51.440137 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:51.442737 (+ 2600us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":154,"threads_started":1}
W20260926 16:32:51.444064 20469 heartbeater.cc:644] Failed to heartbeat to 127.11.8.254:36337 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.8.254:36337: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Sep 26 16:32:51 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.195@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:32:51 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.195@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:32:51.449159 20471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:51.444478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.195:49225)
0926 16:32:51.444562 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:51.444576 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:51.444670 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:51.444892 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:51.444895 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:51.445046 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:51.445126 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:51.445130 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:51.445820 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:51.445826 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:51.446414 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:51.446423 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:51.446841 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:51.446844 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:51.446934 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:51.447378 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:51.447394 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:51.449032 (+ 1638us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":25}
W20260926 16:32:51.449254 20204 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:51.444567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:49225 (local address 127.11.8.254:36337)
0926 16:32:51.444705 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:51.444711 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:51.444725 (+ 14us) server_negotiation.cc:407] Connection header received
0926 16:32:51.444757 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:51.444760 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:51.444798 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:51.444878 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:51.445232 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:51.445716 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:51.446527 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:51.446737 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:51.446839 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:51.449134 (+ 2295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.195:49225: BlockingRecv error: recv got EOF from 127.11.8.195:49225 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 16:32:51.450646 20340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:51.441944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43332 (local address 127.11.8.254:36337)
0926 16:32:51.442067 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:51.442071 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:51.442084 (+ 13us) server_negotiation.cc:407] Connection header received
0926 16:32:51.442187 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:51.442190 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:51.442227 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:51.442357 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:51.443216 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:51.443759 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:51.444819 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:51.445049 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:51.445153 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:51.446577 (+ 1424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:32:51.446583 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:32:51.446594 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:32:51.446619 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:32:51.448498 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:51.448989 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:51.448994 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:51.448998 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:51.449110 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:32:51.449408 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:32:51.449410 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:32:51.449411 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:32:51.449716 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:32:51.449917 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0926 16:32:51.450227 (+ 310us) server_negotiation.cc:299] Negotiation successful
0926 16:32:51.450480 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
Sep 26 16:32:51 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.195@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:32:51 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.195@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:32:51.455782 20471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:51.450320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.195:55955)
0926 16:32:51.450476 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:51.450490 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:51.450557 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:51.451448 (+ 891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:51.451451 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:51.451573 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:51.451637 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:51.451641 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:51.452534 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:51.452541 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:51.453231 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:51.453240 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:51.453657 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:51.453661 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:51.453737 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:51.454116 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:51.454127 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:51.455706 (+ 1579us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":83}
W20260926 16:32:51.455971 20204 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:51.450402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:55955 (local address 127.11.8.254:36337)
0926 16:32:51.450523 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:51.450526 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:51.450536 (+ 10us) server_negotiation.cc:407] Connection header received
0926 16:32:51.450631 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:51.450633 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:51.450662 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:51.450762 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:51.451743 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:51.452431 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:51.453349 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:51.453549 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:51.453655 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:51.455855 (+ 2200us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.195:55955: BlockingRecv error: recv got EOF from 127.11.8.195:55955 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260926 16:32:51.456158 20469 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 26 16:32:52 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.193@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:32:52 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.193@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:32:52.106804 20473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:52.094079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.193:52643)
0926 16:32:52.099820 (+ 5741us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:52.099869 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:52.100019 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:52.100419 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:52.100424 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:52.100749 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:52.100879 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:52.100888 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:52.102124 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:52.102127 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:52.103002 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:52.103013 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:52.103674 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:52.103678 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:52.103775 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:52.104425 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:52.104445 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:52.106607 (+ 2162us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":5643,"thread_start_us":76,"threads_started":1}
W20260926 16:32:52.106849 20474 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:52.094447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:52643 (local address 127.11.8.254:36337)
0926 16:32:52.094722 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:52.094727 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:52.099960 (+ 5233us) server_negotiation.cc:407] Connection header received
0926 16:32:52.100133 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:52.100138 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:52.100232 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:52.100345 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:52.101016 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:52.101996 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:52.103153 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:52.103538 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:52.103665 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:52.106744 (+ 3079us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.193:52643: BlockingRecv error: recv got EOF from 127.11.8.193:52643 (error 108)
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":109,"threads_started":1}
Sep 26 16:32:52 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.194@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:32:52 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.194@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:32:52.279323 20475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:52.274243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.194:39705)
0926 16:32:52.274523 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:52.274537 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:52.274632 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:52.274862 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:52.274864 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:52.275058 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:52.275151 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:52.275156 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:52.275857 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:52.275859 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:52.276427 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:52.276435 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:52.276839 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:52.276841 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:52.276902 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:52.277377 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:52.277388 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:52.279205 (+ 1817us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":115,"threads_started":1}
W20260926 16:32:52.279491 20474 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:52.274332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:39705 (local address 127.11.8.254:36337)
0926 16:32:52.274486 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:52.274490 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:52.274610 (+ 120us) server_negotiation.cc:407] Connection header received
0926 16:32:52.274712 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:52.274715 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:52.274758 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:52.274828 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:52.275252 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:52.275766 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:52.276530 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:52.276742 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:52.276819 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:52.279336 (+ 2517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.194:39705: BlockingRecv error: recv got EOF from 127.11.8.194:39705 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 26 16:32:52 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.195@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:32:52 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.195@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:32:52.462340 20476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:52.456729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.195:34749)
0926 16:32:52.457050 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:52.457067 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:52.457176 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:52.457465 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:52.457468 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:52.457681 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:52.457794 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:52.457800 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:52.458645 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:52.458649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:52.459243 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:52.459251 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:52.459725 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:52.459728 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:52.459828 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:52.460318 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:52.460329 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:52.462167 (+ 1838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":107,"threads_started":1}
W20260926 16:32:52.462502 20474 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:52.456969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:34749 (local address 127.11.8.254:36337)
0926 16:32:52.457137 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:52.457141 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:52.457156 (+ 15us) server_negotiation.cc:407] Connection header received
0926 16:32:52.457295 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:52.457298 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:52.457351 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:52.457438 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:52.457921 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:52.458534 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:52.459372 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:52.459603 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:52.459707 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:52.462356 (+ 2649us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.195:34749: BlockingRecv error: recv got EOF from 127.11.8.195:34749 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Sep 26 16:32:53 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.193@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:32:53 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.193@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:32:53.112886 20477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:53.107751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.193:35977)
0926 16:32:53.108029 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:53.108042 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:53.108131 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:53.108384 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:53.108386 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:53.108564 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:53.108658 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:53.108663 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:53.109416 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:53.109419 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:53.109984 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:53.109991 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:53.110476 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:53.110479 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:53.110540 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:53.111028 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:53.111040 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:53.112761 (+ 1721us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":72,"threads_started":1}
W20260926 16:32:53.113032 20478 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:53.107847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:35977 (local address 127.11.8.254:36337)
0926 16:32:53.108097 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:53.108101 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:53.108114 (+ 13us) server_negotiation.cc:407] Connection header received
0926 16:32:53.108199 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:53.108201 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:53.108242 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:53.108319 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:53.108763 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:53.109292 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:53.110084 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:53.110362 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:53.110475 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:53.112889 (+ 2414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.193:35977: BlockingRecv error: recv got EOF from 127.11.8.193:35977 (error 108)
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":82,"threads_started":1}
Sep 26 16:32:53 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.194@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:32:53 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.194@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:32:53.285176 20479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:53.280094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.194:50541)
0926 16:32:53.280363 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:53.280376 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:53.280479 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:53.280693 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:53.280695 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:53.280863 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:53.280956 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:53.280961 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:53.281686 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:53.281688 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:53.282296 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:53.282304 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:53.282718 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:53.282721 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:53.282788 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:53.283268 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:53.283280 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:53.285046 (+ 1766us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":106,"threads_started":1}
W20260926 16:32:53.285293 20478 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:53.280140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:50541 (local address 127.11.8.254:36337)
0926 16:32:53.280283 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:53.280288 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:53.280429 (+ 141us) server_negotiation.cc:407] Connection header received
0926 16:32:53.280557 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:53.280559 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:53.280600 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:53.280657 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:53.281053 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:53.281576 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:53.282379 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:53.282593 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:53.282669 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:53.285181 (+ 2512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.194:50541: BlockingRecv error: recv got EOF from 127.11.8.194:50541 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 26 16:32:53 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.195@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:32:53 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.195@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:32:53.468684 20480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:53.463138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.195:51743)
0926 16:32:53.463471 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:53.463486 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:53.463587 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:53.463821 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:53.463823 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:53.464000 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:53.464096 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:53.464100 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:53.464849 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:53.464852 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:53.465410 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:53.465418 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:53.465907 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:53.465912 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:53.465984 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:53.466578 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:53.466590 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:53.468573 (+ 1983us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":105,"threads_started":1}
W20260926 16:32:53.468880 20478 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:53.463325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:51743 (local address 127.11.8.254:36337)
0926 16:32:53.463473 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:53.463477 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:53.463539 (+ 62us) server_negotiation.cc:407] Connection header received
0926 16:32:53.463653 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:53.463656 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:53.463712 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:53.463779 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:53.464203 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:53.464739 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:53.465533 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:53.465756 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:53.465848 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:53.468713 (+ 2865us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.195:51743: BlockingRecv error: recv got EOF from 127.11.8.195:51743 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Sep 26 16:32:54 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.193@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:32:54 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.193@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:32:54.119300 20481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:54.113693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.193:39525)
0926 16:32:54.113995 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:54.114008 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:54.114100 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:54.114401 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:54.114403 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:54.114617 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:54.114715 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:54.114720 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:54.115683 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:54.115686 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:54.116309 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:54.116318 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:54.116784 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:54.116787 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:54.116859 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:54.117384 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:54.117395 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:54.119183 (+ 1788us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":89,"threads_started":1}
W20260926 16:32:54.119531 20482 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:54.113817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:39525 (local address 127.11.8.254:36337)
0926 16:32:54.114069 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:54.114073 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:54.114087 (+ 14us) server_negotiation.cc:407] Connection header received
0926 16:32:54.114189 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:54.114192 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:54.114236 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:54.114365 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:54.114899 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:54.115580 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:54.116429 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:54.116671 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:54.116757 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:54.119367 (+ 2610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.193:39525: BlockingRecv error: recv got EOF from 127.11.8.193:39525 (error 108)
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":56,"threads_started":1}
Sep 26 16:32:54 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.194@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:32:54 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.194@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:32:54.291590 20483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:54.286052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.194:46871)
0926 16:32:54.286390 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:54.286408 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:54.286552 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:54.286820 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:54.286823 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:54.287031 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:54.287119 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:54.287125 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:54.287960 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:54.287965 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:54.288526 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:54.288533 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:54.288995 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:54.288999 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:54.289084 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:54.289558 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:54.289569 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:54.291422 (+ 1853us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":88,"threads_started":1}
W20260926 16:32:54.291644 20482 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:54.286144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:46871 (local address 127.11.8.254:36337)
0926 16:32:54.286350 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:54.286355 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:54.286490 (+ 135us) server_negotiation.cc:407] Connection header received
0926 16:32:54.286651 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:54.286656 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:54.286710 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:54.286807 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:54.287247 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:54.287776 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:54.288648 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:54.288885 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:54.288994 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:54.291539 (+ 2545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.194:46871: BlockingRecv error: recv got EOF from 127.11.8.194:46871 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Sep 26 16:32:54 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.195@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:32:54 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.195@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:32:54.475042 20484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:54.469585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.195:60549)
0926 16:32:54.469882 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:54.469897 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:54.470036 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:54.470374 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:54.470378 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:54.470606 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:54.470718 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:54.470724 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:54.471441 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:54.471443 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:54.472016 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:54.472023 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:54.472474 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:54.472477 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:54.472544 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:54.473123 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:54.473135 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:54.474905 (+ 1770us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":96,"threads_started":1}
W20260926 16:32:54.475271 20482 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:54.469654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:60549 (local address 127.11.8.254:36337)
0926 16:32:54.469819 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:54.469825 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:54.469989 (+ 164us) server_negotiation.cc:407] Connection header received
0926 16:32:54.470134 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:54.470138 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:54.470190 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:54.470315 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:54.470822 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:54.471346 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:54.472119 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:54.472344 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:54.472423 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:54.475042 (+ 2619us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.195:60549: BlockingRecv error: recv got EOF from 127.11.8.195:60549 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Sep 26 16:32:55 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.193@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:32:55 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.193@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:32:55.131242 20485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:55.120047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.193:52499)
0926 16:32:55.120388 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:55.120419 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:55.120520 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:55.120807 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:55.120810 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:55.120997 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:55.121108 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:55.121114 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:55.122088 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:55.122090 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:55.122740 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:55.122750 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:55.123242 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:55.123245 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:55.123318 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:55.128632 (+ 5314us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:55.128653 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:55.131067 (+ 2414us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":112,"threads_started":1}
W20260926 16:32:55.131327 20486 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:55.120146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:52499 (local address 127.11.8.254:36337)
0926 16:32:55.120417 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:55.120421 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:55.120486 (+ 65us) server_negotiation.cc:407] Connection header received
0926 16:32:55.120603 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:55.120606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:55.120654 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:55.120737 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:55.121223 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:55.121983 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:55.122873 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:55.123124 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:55.123213 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:55.131199 (+ 7986us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.193:52499: BlockingRecv error: recv got EOF from 127.11.8.193:52499 (error 108)
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":65,"threads_started":1}
Sep 26 16:32:55 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.194@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:32:55 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.194@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:32:55.298251 20487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:55.292347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.194:37327)
0926 16:32:55.292697 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:55.292714 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:55.292858 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:55.293156 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:55.293158 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:55.293365 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:55.293451 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:55.293457 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:55.294212 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:55.294217 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:55.294867 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:55.294876 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:55.295370 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:55.295374 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:55.295452 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:55.296031 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:55.296042 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:55.298095 (+ 2053us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":103,"threads_started":1}
W20260926 16:32:55.298405 20486 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:55.292476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:37327 (local address 127.11.8.254:36337)
0926 16:32:55.292663 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:55.292668 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:55.292799 (+ 131us) server_negotiation.cc:407] Connection header received
0926 16:32:55.292956 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:55.292959 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:55.293041 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:55.293133 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:55.293571 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:55.294101 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:55.295010 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:55.295255 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:55.295358 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:55.298245 (+ 2887us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.194:37327: BlockingRecv error: recv got EOF from 127.11.8.194:37327 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Sep 26 16:32:55 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.195@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:32:55 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.195@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:32:55.481175 20488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:55.475750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.195:49149)
0926 16:32:55.476020 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:55.476038 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:55.476142 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:55.476371 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:55.476374 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:55.476552 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:55.476646 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:55.476651 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:55.477396 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:55.477399 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:55.477958 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:55.477968 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:55.478459 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:55.478462 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:55.478533 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:55.479020 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:55.479032 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:55.481029 (+ 1997us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":103,"threads_started":1}
W20260926 16:32:55.481498 20486 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:55.475811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:49149 (local address 127.11.8.254:36337)
0926 16:32:55.475954 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:55.475958 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:55.476116 (+ 158us) server_negotiation.cc:407] Connection header received
0926 16:32:55.476226 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:55.476228 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:55.476273 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:55.476334 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:55.476744 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:55.477283 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:55.478076 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:55.478305 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:55.478416 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:55.481195 (+ 2779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.195:49149: BlockingRecv error: recv got EOF from 127.11.8.195:49149 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 26 16:32:56 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.193@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:32:56 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.193@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:32:56.138635 20489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:56.132157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.193:39547)
0926 16:32:56.132423 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:56.132439 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:56.132573 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:56.132863 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:56.132869 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:56.133085 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:56.133197 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:56.133203 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:56.134392 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:56.134398 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:56.135391 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:56.135405 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:56.135986 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:56.135990 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:56.136074 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:56.136607 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:56.136618 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:56.138485 (+ 1867us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":84,"threads_started":1}
W20260926 16:32:56.138865 20490 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:56.132225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:39547 (local address 127.11.8.254:36337)
0926 16:32:56.132527 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:56.132532 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:56.132548 (+ 16us) server_negotiation.cc:407] Connection header received
0926 16:32:56.132660 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:56.132663 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:56.132744 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:56.132847 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:56.133305 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:56.134215 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:56.135508 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:56.135856 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:56.135966 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:56.138649 (+ 2683us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.193:39547: BlockingRecv error: recv got EOF from 127.11.8.193:39547 (error 108)
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":90,"threads_started":1}
Sep 26 16:32:56 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.194@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:32:56 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.194@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:32:56.304617 20491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:56.299075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.194:42185)
0926 16:32:56.299496 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:56.299513 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:56.299650 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:56.299925 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:56.299928 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:56.300136 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:56.300224 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:56.300229 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:56.300978 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:56.300982 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:56.301556 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:56.301564 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:56.302029 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:56.302033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:56.302119 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:56.302675 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:56.302687 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:56.304418 (+ 1731us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":107,"threads_started":1}
W20260926 16:32:56.304641 20490 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:56.299233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:42185 (local address 127.11.8.254:36337)
0926 16:32:56.299485 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:56.299489 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:56.299594 (+ 105us) server_negotiation.cc:407] Connection header received
0926 16:32:56.299759 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:56.299762 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:56.299815 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:56.299898 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:56.300341 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:56.300871 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:56.301678 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:56.301910 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:56.302027 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:56.304534 (+ 2507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.194:42185: BlockingRecv error: recv got EOF from 127.11.8.194:42185 (error 108)
Metrics: {"server-negotiator.queue_time_us":160}
Sep 26 16:32:56 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.195@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:32:56 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.195@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:32:56.487545 20492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:56.481929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.195:45549)
0926 16:32:56.482296 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:56.482309 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:56.482450 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:56.482744 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:56.482747 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:56.482961 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:56.483049 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:56.483055 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:56.483826 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:56.483831 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:56.484415 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:56.484424 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:56.484900 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:56.484904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:56.484990 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:56.485471 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:56.485483 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:56.487369 (+ 1886us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":104,"threads_started":1}
W20260926 16:32:56.487612 20490 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:56.482055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:45549 (local address 127.11.8.254:36337)
0926 16:32:56.482227 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:56.482232 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:56.482387 (+ 155us) server_negotiation.cc:407] Connection header received
0926 16:32:56.482574 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:56.482578 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:56.482635 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:56.482727 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:56.483170 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:56.483715 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:56.484548 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:56.484783 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:56.484905 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:56.487501 (+ 2596us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.195:45549: BlockingRecv error: recv got EOF from 127.11.8.195:45549 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Sep 26 16:32:57 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.193@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:32:57 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.193@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:32:57.144518 20493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:57.139333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.193:56371)
0926 16:32:57.139599 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:57.139612 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:57.139703 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:57.139899 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:57.139902 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:57.140093 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:57.140183 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:57.140188 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:57.140930 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:57.140932 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:57.141486 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:57.141493 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:57.141990 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:57.141993 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:57.142055 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:57.142572 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:57.142582 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:57.144404 (+ 1822us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":86,"threads_started":1}
W20260926 16:32:57.144737 20494 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:57.139390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:56371 (local address 127.11.8.254:36337)
0926 16:32:57.139668 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:57.139672 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:57.139685 (+ 13us) server_negotiation.cc:407] Connection header received
0926 16:32:57.139771 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:57.139773 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:57.139812 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:57.139886 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:57.140289 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:57.140818 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:57.141611 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:57.141877 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:57.141969 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:57.144550 (+ 2581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.193:56371: BlockingRecv error: recv got EOF from 127.11.8.193:56371 (error 108)
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":76,"threads_started":1}
Sep 26 16:32:57 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.194@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:32:57 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.194@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:32:57.311196 20495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:57.305528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.194:56513)
0926 16:32:57.305802 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:57.305815 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:57.305910 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:57.306219 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:57.306223 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:57.306442 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:57.306540 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:57.306545 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:57.307659 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:57.307664 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:57.308376 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:57.308385 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:57.308786 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:57.308788 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:57.308845 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:57.309328 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:57.309339 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:57.311086 (+ 1747us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":98,"threads_started":1}
W20260926 16:32:57.311333 20494 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:57.305598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:56513 (local address 127.11.8.254:36337)
0926 16:32:57.305723 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:57.305726 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:57.305868 (+ 142us) server_negotiation.cc:407] Connection header received
0926 16:32:57.305996 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:57.305999 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:57.306044 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:57.306107 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:57.306692 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:57.307519 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:57.308479 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:57.308694 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:57.308771 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:57.311217 (+ 2446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.194:56513: BlockingRecv error: recv got EOF from 127.11.8.194:56513 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Sep 26 16:32:57 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.195@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:32:57 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.195@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:32:57.494639 20496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:57.488500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.195:47107)
0926 16:32:57.488750 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:57.488763 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:57.488856 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:57.489125 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:57.489128 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:57.489431 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:57.489558 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:57.489565 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:57.490533 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:57.490539 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:57.491172 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:57.491180 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:57.491678 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:57.491682 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:57.491776 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:57.492362 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:57.492387 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:57.494431 (+ 2044us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":85,"threads_started":1}
W20260926 16:32:57.494678 20494 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:57.488599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:47107 (local address 127.11.8.254:36337)
0926 16:32:57.488762 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:57.488769 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:57.488815 (+ 46us) server_negotiation.cc:407] Connection header received
0926 16:32:57.488944 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:57.488947 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:57.488994 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:57.489098 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:57.489684 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:57.490421 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:57.491305 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:57.491557 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:57.491676 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:57.494570 (+ 2894us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.195:47107: BlockingRecv error: recv got EOF from 127.11.8.195:47107 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Sep 26 16:32:58 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.193@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:32:58 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.193@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:32:58.150955 20497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:58.145258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.193:43133)
0926 16:32:58.145569 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:58.145582 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:58.145671 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:58.145865 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:58.145867 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:58.146064 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:58.146158 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:58.146162 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:58.146970 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:58.146974 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:58.147832 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:58.147843 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:58.148341 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:58.148344 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:58.148426 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:58.148966 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:58.148981 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:58.150840 (+ 1859us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":70,"threads_started":1}
W20260926 16:32:58.151139 20498 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:58.145332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:43133 (local address 127.11.8.254:36337)
0926 16:32:58.145610 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:58.145613 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:58.145627 (+ 14us) server_negotiation.cc:407] Connection header received
0926 16:32:58.145738 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:58.145740 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:58.145780 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:58.145854 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:58.146255 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:58.146851 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:58.147961 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:58.148223 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:58.148334 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:58.150972 (+ 2638us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.193:43133: BlockingRecv error: recv got EOF from 127.11.8.193:43133 (error 108)
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":89,"threads_started":1}
Sep 26 16:32:58 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.194@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:32:58 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.194@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:32:58.317914 20499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:58.312007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.194:53755)
0926 16:32:58.312326 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:58.312341 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:58.312459 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:58.312755 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:58.312762 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:58.313045 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:58.313189 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:58.313203 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:58.314180 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:58.314186 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:58.315063 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:58.315072 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:58.315472 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:58.315474 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:58.315554 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:58.316058 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:58.316069 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:58.317805 (+ 1736us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":84,"threads_started":1}
W20260926 16:32:58.318071 20498 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:58.312113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:53755 (local address 127.11.8.254:36337)
0926 16:32:58.312254 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:58.312260 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:58.312416 (+ 156us) server_negotiation.cc:407] Connection header received
0926 16:32:58.312576 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:58.312580 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:58.312631 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:58.312710 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:58.313299 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:58.314051 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:58.315164 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:58.315379 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:58.315488 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:58.317928 (+ 2440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.194:53755: BlockingRecv error: recv got EOF from 127.11.8.194:53755 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 26 16:32:58 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.195@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:32:58 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.195@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:32:58.505664 20500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:58.495470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.195:40527)
0926 16:32:58.495821 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:58.495835 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:58.495931 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:58.496169 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:58.496172 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:58.496397 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:58.496496 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:58.496502 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:58.497431 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:58.497435 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:58.498175 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:58.498187 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:58.498656 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:58.498659 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:58.498718 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:58.503521 (+ 4803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:58.503534 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:58.505513 (+ 1979us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":122,"threads_started":1}
W20260926 16:32:58.505775 20498 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:58.495560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:40527 (local address 127.11.8.254:36337)
0926 16:32:58.495687 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:58.495691 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:58.495899 (+ 208us) server_negotiation.cc:407] Connection header received
0926 16:32:58.496007 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:58.496009 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:58.496056 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:58.496125 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:58.496608 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:58.497276 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:58.498328 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:58.498554 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:58.498635 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:58.505664 (+ 7029us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.195:40527: BlockingRecv error: recv got EOF from 127.11.8.195:40527 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 26 16:32:59 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.193@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:32:59 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.193@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:32:59.157174 20501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:59.151748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.193:49229)
0926 16:32:59.152053 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:59.152066 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:59.152162 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:59.152402 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:59.152404 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:59.152583 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:59.152680 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:59.152686 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:59.153413 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:59.153416 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:59.153986 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:59.153994 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:59.154457 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:59.154460 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:59.154522 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:59.154980 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:59.154991 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:59.156999 (+ 2008us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":88,"threads_started":1}
W20260926 16:32:59.157302 20502 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:59.151860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:49229 (local address 127.11.8.254:36337)
0926 16:32:59.152090 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:59.152093 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:59.152110 (+ 17us) server_negotiation.cc:407] Connection header received
0926 16:32:59.152247 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:59.152249 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:59.152291 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:59.152363 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:59.152775 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:59.153315 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:59.154085 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:59.154363 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:59.154441 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:59.157161 (+ 2720us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.193:49229: BlockingRecv error: recv got EOF from 127.11.8.193:49229 (error 108)
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":75,"threads_started":1}
Sep 26 16:32:59 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.194@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:32:59 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.194@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:32:59.324359 20503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:59.318722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.194:48205)
0926 16:32:59.319007 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:59.319023 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:59.319120 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:59.319331 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:59.319333 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:59.319505 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:59.319606 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:59.319611 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:59.320494 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:59.320496 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:59.321122 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:59.321131 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:59.321526 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:59.321528 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:59.321585 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:59.322329 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:59.322342 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:59.324191 (+ 1849us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":137,"threads_started":1}
W20260926 16:32:59.324401 20502 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:59.318773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:48205 (local address 127.11.8.254:36337)
0926 16:32:59.318931 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:59.318936 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:59.319092 (+ 156us) server_negotiation.cc:407] Connection header received
0926 16:32:59.319186 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:59.319188 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:59.319232 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:59.319299 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:59.319720 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:59.320405 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:59.321219 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:59.321428 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:59.321504 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:59.324303 (+ 2799us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.194:48205: BlockingRecv error: recv got EOF from 127.11.8.194:48205 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Sep 26 16:32:59 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.195@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:32:59 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.195@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:32:59.512089 20504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:32:59.506505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.195:39783)
0926 16:32:59.506760 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0926 16:32:59.506774 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:32:59.506880 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:32:59.507162 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:32:59.507165 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:32:59.507353 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:32:59.507462 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:59.507468 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:59.508211 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:59.508214 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:59.508936 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:32:59.508946 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:32:59.509469 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:32:59.509472 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:32:59.509559 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:59.510053 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:32:59.510065 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:32:59.511926 (+ 1861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":70,"threads_started":1}
W20260926 16:32:59.512178 20502 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:32:59.506587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:39783 (local address 127.11.8.254:36337)
0926 16:32:59.506774 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0926 16:32:59.506778 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:32:59.506851 (+ 73us) server_negotiation.cc:407] Connection header received
0926 16:32:59.506963 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:32:59.506966 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:32:59.507011 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:32:59.507078 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:32:59.507562 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:59.508094 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:59.509071 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:32:59.509346 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:32:59.509451 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:32:59.512060 (+ 2609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.195:39783: BlockingRecv error: recv got EOF from 127.11.8.195:39783 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Sep 26 16:33:00 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.193@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:33:00 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.193@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:33:00.169495 20505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:00.157889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.193:49277)
0926 16:33:00.158162 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:00.158177 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:33:00.158307 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:33:00.158672 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:33:00.158674 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:33:00.158855 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:33:00.158949 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:00.158954 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:00.159745 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:00.159747 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:00.160292 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:00.160299 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:00.160706 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:00.160708 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:00.160767 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:00.167230 (+ 6463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:33:00.167243 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:33:00.169266 (+ 2023us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":124,"threads_started":1}
W20260926 16:33:00.169693 20506 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:33:00.158221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:49277 (local address 127.11.8.254:36337)
0926 16:33:00.158472 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:00.158476 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:00.158487 (+ 11us) server_negotiation.cc:407] Connection header received
0926 16:33:00.158518 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:00.158520 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:00.158559 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:00.158632 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:33:00.159078 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:00.159646 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:00.160389 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:00.160609 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:00.160684 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:00.169431 (+ 8747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.193:49277: BlockingRecv error: recv got EOF from 127.11.8.193:49277 (error 108)
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":100,"threads_started":1}
Sep 26 16:33:00 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.194@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:33:00 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.194@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:33:00.330909 20507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:00.325207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.194:34099)
0926 16:33:00.325480 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:00.325493 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:33:00.325593 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:33:00.325860 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:33:00.325863 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:33:00.326044 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:33:00.326137 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:00.326143 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:00.326980 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:00.326983 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:00.327727 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:00.327736 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:00.328306 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:00.328310 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:00.328392 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:00.328883 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:33:00.328894 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:33:00.330724 (+ 1830us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":98,"threads_started":1}
W20260926 16:33:00.330973 20506 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:33:00.325262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:34099 (local address 127.11.8.254:36337)
0926 16:33:00.325404 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:00.325407 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:00.325596 (+ 189us) server_negotiation.cc:407] Connection header received
0926 16:33:00.325661 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:00.325663 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:00.325709 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:00.325789 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:33:00.326245 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:00.326845 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:00.327843 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:00.328178 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:00.328295 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:00.330854 (+ 2559us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.194:34099: BlockingRecv error: recv got EOF from 127.11.8.194:34099 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 26 16:33:00 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.195@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:33:00 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.195@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:33:00.518435 20508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:00.512924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.195:42539)
0926 16:33:00.513232 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:00.513244 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:33:00.513379 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:33:00.513647 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:33:00.513649 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:33:00.513821 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:33:00.513915 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:00.513921 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:00.514688 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:00.514691 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:00.515238 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:00.515246 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:00.515726 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:00.515729 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:00.515792 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:00.516351 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:33:00.516363 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:33:00.518282 (+ 1919us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":79,"threads_started":1}
W20260926 16:33:00.518618 20506 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:33:00.512998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:42539 (local address 127.11.8.254:36337)
0926 16:33:00.513150 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:00.513154 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:00.513344 (+ 190us) server_negotiation.cc:407] Connection header received
0926 16:33:00.513489 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:00.513492 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:00.513542 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:00.513609 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:33:00.514039 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:00.514590 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:00.515349 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:00.515628 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:00.515706 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:00.518480 (+ 2774us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.195:42539: BlockingRecv error: recv got EOF from 127.11.8.195:42539 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 26 16:33:01 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.193@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:33:01 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.193@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:33:01.176277 20509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:01.170384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.193:48693)
0926 16:33:01.170735 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:01.170751 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:33:01.170867 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:33:01.171182 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:33:01.171185 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:33:01.171444 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:33:01.171551 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:01.171556 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:01.172383 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:01.172389 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:01.173026 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:01.173034 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:01.173540 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:01.173544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:01.173653 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:01.174200 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:33:01.174212 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:33:01.176164 (+ 1952us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":98,"threads_started":1}
W20260926 16:33:01.176496 20510 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:33:01.170415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:48693 (local address 127.11.8.254:36337)
0926 16:33:01.170728 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:01.170733 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:01.170817 (+ 84us) server_negotiation.cc:407] Connection header received
0926 16:33:01.170986 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:01.170990 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:01.171055 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:01.171159 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:33:01.171681 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:01.172265 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:01.173157 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:01.173413 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:01.173541 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:01.176304 (+ 2763us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.193:48693: BlockingRecv error: recv got EOF from 127.11.8.193:48693 (error 108)
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":80,"threads_started":1}
Sep 26 16:33:01 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.194@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:33:01 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.194@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:33:01.337435 20511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:01.331723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.194:41617)
0926 16:33:01.332071 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:01.332107 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 16:33:01.332219 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:33:01.332532 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:33:01.332535 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:33:01.332731 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:33:01.332819 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:01.332825 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:01.333671 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:01.333675 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:01.334319 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:01.334327 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:01.334769 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:01.334773 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:01.334857 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:01.335365 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:33:01.335376 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:33:01.337263 (+ 1887us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":105,"threads_started":1}
W20260926 16:33:01.337513 20510 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:33:01.331905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:41617 (local address 127.11.8.254:36337)
0926 16:33:01.332075 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:01.332080 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:01.332187 (+ 107us) server_negotiation.cc:407] Connection header received
0926 16:33:01.332330 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:01.332333 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:01.332395 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:01.332487 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:33:01.332935 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:01.333567 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:01.334430 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:01.334659 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:01.334768 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:01.337402 (+ 2634us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.194:41617: BlockingRecv error: recv got EOF from 127.11.8.194:41617 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 26 16:33:01 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.195@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:33:01 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.195@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:33:01.525297 20512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:01.519197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.195:49765)
0926 16:33:01.519459 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:01.519474 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:33:01.519567 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:33:01.519807 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:33:01.519809 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:33:01.519994 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:33:01.520091 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:01.520096 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:01.520929 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:01.520932 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:01.522032 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:01.522044 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:01.522681 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:01.522685 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:01.522751 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:01.523267 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:33:01.523280 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:33:01.525125 (+ 1845us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":98,"threads_started":1}
W20260926 16:33:01.525334 20510 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:33:01.519365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:49765 (local address 127.11.8.254:36337)
0926 16:33:01.519502 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:01.519506 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:01.519523 (+ 17us) server_negotiation.cc:407] Connection header received
0926 16:33:01.519653 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:01.519655 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:01.519700 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:01.519774 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:33:01.520187 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:01.520778 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:01.522194 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:01.522538 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:01.522633 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:01.525238 (+ 2605us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.195:49765: BlockingRecv error: recv got EOF from 127.11.8.195:49765 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 26 16:33:02 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.193@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:33:02 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.193@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:33:02.182919 20513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:02.177060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.193:38909)
0926 16:33:02.177353 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:02.177366 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:33:02.177475 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:33:02.177734 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:33:02.177736 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:33:02.177911 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:33:02.178011 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:02.178016 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:02.178877 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:02.178881 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:02.179463 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:02.179471 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:02.179984 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:02.179987 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:02.180053 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:02.180539 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:33:02.180551 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:33:02.182792 (+ 2241us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":86,"threads_started":1}
W20260926 16:33:02.183050 20514 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:33:02.177202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:38909 (local address 127.11.8.254:36337)
0926 16:33:02.177430 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:02.177433 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:02.177447 (+ 14us) server_negotiation.cc:407] Connection header received
0926 16:33:02.177572 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:02.177574 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:02.177622 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:02.177700 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:33:02.178132 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:02.178739 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:02.179603 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:02.179828 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:02.179917 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:02.182921 (+ 3004us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.193:38909: BlockingRecv error: recv got EOF from 127.11.8.193:38909 (error 108)
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":58,"threads_started":1}
Sep 26 16:33:02 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.194@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:33:02 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.194@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:33:02.343494 20515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:02.338152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.194:34255)
0926 16:33:02.338423 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:02.338436 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:33:02.338536 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:33:02.338768 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:33:02.338770 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:33:02.338952 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:33:02.339074 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:02.339079 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:02.339776 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:02.339778 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:02.340336 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:02.340342 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:02.340732 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:02.340735 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:02.340792 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:02.341265 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:33:02.341276 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:33:02.343330 (+ 2054us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":69,"threads_started":1}
W20260926 16:33:02.343549 20514 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:33:02.338235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:34255 (local address 127.11.8.254:36337)
0926 16:33:02.338416 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:02.338423 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:02.338507 (+ 84us) server_negotiation.cc:407] Connection header received
0926 16:33:02.338616 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:02.338618 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:02.338663 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:02.338729 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:33:02.339166 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:02.339683 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:02.340429 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:02.340641 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:02.340715 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:02.343444 (+ 2729us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.194:34255: BlockingRecv error: recv got EOF from 127.11.8.194:34255 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Sep 26 16:33:02 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.195@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:33:02 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.195@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:33:02.531584 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:02.526228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.195:59135)
0926 16:33:02.526492 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:02.526505 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:33:02.526602 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:33:02.526829 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:33:02.526831 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:33:02.527002 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:33:02.527118 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:02.527123 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:02.527867 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:02.527869 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:02.528458 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:02.528466 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:02.528895 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:02.528898 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:02.528964 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:02.529528 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:33:02.529540 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:33:02.531454 (+ 1914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":116,"threads_started":1}
W20260926 16:33:02.531778 20514 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:33:02.526247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:59135 (local address 127.11.8.254:36337)
0926 16:33:02.526397 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:02.526402 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:02.526579 (+ 177us) server_negotiation.cc:407] Connection header received
0926 16:33:02.526680 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:02.526683 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:02.526730 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:02.526796 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:33:02.527215 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:02.527769 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:02.528571 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:02.528793 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:02.528872 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:02.531612 (+ 2740us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.195:59135: BlockingRecv error: recv got EOF from 127.11.8.195:59135 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 26 16:33:03 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.193@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:33:03 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.193@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:33:03.188736 20517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:03.183705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.193:43329)
0926 16:33:03.183946 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:03.183959 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:33:03.184076 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:33:03.184299 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:33:03.184301 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:33:03.184471 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:33:03.184564 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:03.184569 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:03.185290 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:03.185293 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:03.185856 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:03.185864 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:03.186314 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:03.186323 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:03.186384 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:03.186853 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:33:03.186863 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:33:03.188625 (+ 1762us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":85,"threads_started":1}
W20260926 16:33:03.188951 20518 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:33:03.183751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:43329 (local address 127.11.8.254:36337)
0926 16:33:03.183980 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:03.183983 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:03.184027 (+ 44us) server_negotiation.cc:407] Connection header received
0926 16:33:03.184165 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:03.184167 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:03.184213 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:03.184286 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:33:03.184656 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:03.185184 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:03.185960 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:03.186179 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:03.186254 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:03.188796 (+ 2542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.193:43329: BlockingRecv error: recv got EOF from 127.11.8.193:43329 (error 108)
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":62,"threads_started":1}
Sep 26 16:33:03 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.194@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:33:03 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.194@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:33:03.350044 20519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:03.344370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.194:51441)
0926 16:33:03.344649 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:03.344686 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 16:33:03.344814 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:33:03.345216 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:33:03.345219 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:33:03.345486 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:33:03.345615 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:03.345623 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:03.346486 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:03.346488 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:03.347051 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:03.347059 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:03.347485 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:03.347488 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:03.347547 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:03.348053 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:33:03.348064 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:33:03.349931 (+ 1867us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":94,"threads_started":1}
W20260926 16:33:03.350323 20518 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:33:03.344465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:51441 (local address 127.11.8.254:36337)
0926 16:33:03.344610 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:03.344614 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:03.344797 (+ 183us) server_negotiation.cc:407] Connection header received
0926 16:33:03.344964 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:03.344967 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:03.345028 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:03.345151 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:33:03.345735 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:03.346364 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:03.347161 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:03.347377 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:03.347458 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:03.350186 (+ 2728us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.194:51441: BlockingRecv error: recv got EOF from 127.11.8.194:51441 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 26 16:33:03 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.195@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:33:03 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.195@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:33:03.537974 20520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:03.532416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.195:41405)
0926 16:33:03.532717 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:03.532730 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:33:03.532872 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:33:03.533146 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:33:03.533149 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:33:03.533357 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:33:03.533448 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:03.533454 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:03.534234 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:03.534240 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:03.534833 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:03.534841 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:03.535334 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:03.535337 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:03.535426 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:03.535991 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:33:03.536004 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:33:03.537853 (+ 1849us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":97,"threads_started":1}
W20260926 16:33:03.538241 20518 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:33:03.532476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:41405 (local address 127.11.8.254:36337)
0926 16:33:03.532667 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:03.532672 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:03.532807 (+ 135us) server_negotiation.cc:407] Connection header received
0926 16:33:03.532973 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:03.532978 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:03.533028 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:03.533122 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:33:03.533583 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:03.534121 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:03.534958 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:03.535217 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:03.535331 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:03.537988 (+ 2657us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.195:41405: BlockingRecv error: recv got EOF from 127.11.8.195:41405 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Sep 26 16:33:04 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.193@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:33:04 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.193@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:33:04.194905 20521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:04.189504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.193:39269)
0926 16:33:04.189764 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:04.189777 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:33:04.189867 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:33:04.190118 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:33:04.190120 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:33:04.190316 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:33:04.190411 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:04.190416 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:04.191167 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:04.191169 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:04.191724 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:04.191732 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:04.192164 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:04.192167 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:04.192225 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:04.192797 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:33:04.192810 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:33:04.194782 (+ 1972us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":90,"threads_started":1}
W20260926 16:33:04.195168 20522 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:33:04.189574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:39269 (local address 127.11.8.254:36337)
0926 16:33:04.189814 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:04.189817 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:04.189831 (+ 14us) server_negotiation.cc:407] Connection header received
0926 16:33:04.189953 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:04.189956 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:04.190004 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:04.190083 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:33:04.190526 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:04.191069 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:04.191826 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:04.192069 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:04.192147 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:04.194927 (+ 2780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.193:39269: BlockingRecv error: recv got EOF from 127.11.8.193:39269 (error 108)
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":96,"threads_started":1}
Sep 26 16:33:04 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.194@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:33:04 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.194@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:33:04.356604 20523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:04.350820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.194:48785)
0926 16:33:04.351130 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:04.351144 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:33:04.351243 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:33:04.351469 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:33:04.351471 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:33:04.351646 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:33:04.351736 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:04.351740 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:04.352493 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:04.352495 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:04.353139 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:04.353148 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:04.353691 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:04.353694 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:04.353772 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:04.354429 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:33:04.354440 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:33:04.356425 (+ 1985us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":101,"threads_started":1}
W20260926 16:33:04.356647 20522 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:33:04.350948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:48785 (local address 127.11.8.254:36337)
0926 16:33:04.351135 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:04.351138 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:04.351209 (+ 71us) server_negotiation.cc:407] Connection header received
0926 16:33:04.351316 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:04.351318 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:04.351359 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:04.351434 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:33:04.351876 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:04.352397 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:04.353288 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:04.353564 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:04.353678 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:04.356544 (+ 2866us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.194:48785: BlockingRecv error: recv got EOF from 127.11.8.194:48785 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Sep 26 16:33:04 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.195@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:33:04 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.195@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:33:04.543884 20524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:04.538750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.195:48259)
0926 16:33:04.539045 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:04.539062 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 16:33:04.539180 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:33:04.539422 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:33:04.539424 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:33:04.539611 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:33:04.539706 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:04.539711 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:04.540410 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:04.540412 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:04.540957 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:04.540964 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:04.541355 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:04.541358 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:04.541420 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:04.541974 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:33:04.541986 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:33:04.543776 (+ 1790us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":109,"threads_started":1}
W20260926 16:33:04.544119 20522 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:33:04.538799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:48259 (local address 127.11.8.254:36337)
0926 16:33:04.538955 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:04.538959 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:04.539146 (+ 187us) server_negotiation.cc:407] Connection header received
0926 16:33:04.539271 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:04.539273 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:04.539319 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:04.539383 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:33:04.539805 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:04.540320 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:04.541053 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:04.541265 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:04.541336 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:04.543958 (+ 2622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.195:48259: BlockingRecv error: recv got EOF from 127.11.8.195:48259 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Sep 26 16:33:05 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.193@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:33:05 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.193@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:33:05.206594 20525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:05.195700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.193:51583)
0926 16:33:05.196001 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:05.196017 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:33:05.196115 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:33:05.196469 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:33:05.196472 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:33:05.196645 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:33:05.196746 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:05.196751 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:05.197500 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:05.197502 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:05.198082 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:05.198091 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:05.198685 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:05.198688 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:05.198754 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:05.204361 (+ 5607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:33:05.204373 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:33:05.206429 (+ 2056us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":81,"threads_started":1}
W20260926 16:33:05.206691 20526 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:33:05.195728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:51583 (local address 127.11.8.254:36337)
0926 16:33:05.196106 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:05.196111 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:05.196130 (+ 19us) server_negotiation.cc:407] Connection header received
0926 16:33:05.196276 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:05.196282 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:05.196334 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:05.196425 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:33:05.196867 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:05.197395 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:05.198230 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:05.198558 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:05.198648 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:05.206556 (+ 7908us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.193:51583: BlockingRecv error: recv got EOF from 127.11.8.193:51583 (error 108)
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":145,"threads_started":1}
Sep 26 16:33:05 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.194@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:33:05 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.194@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:33:05.363163 20527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:05.357443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.194:55719)
0926 16:33:05.357754 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:05.357769 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:33:05.357940 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:33:05.358227 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:33:05.358229 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:33:05.358465 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:33:05.358553 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:05.358559 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:05.359326 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:05.359332 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:05.359927 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:05.359934 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:05.360407 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:05.360410 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:05.360496 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:05.360972 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:33:05.360984 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:33:05.363038 (+ 2054us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":111,"threads_started":1}
W20260926 16:33:05.363451 20526 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:33:05.357514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:55719 (local address 127.11.8.254:36337)
0926 16:33:05.357720 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:05.357724 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:05.357851 (+ 127us) server_negotiation.cc:407] Connection header received
0926 16:33:05.358046 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:05.358051 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:05.358106 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:05.358204 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:33:05.358673 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:05.359217 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:05.360045 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:05.360285 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:05.360398 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:05.363191 (+ 2793us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.194:55719: BlockingRecv error: recv got EOF from 127.11.8.194:55719 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
Sep 26 16:33:05 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.195@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:33:05 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.195@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:33:05.550165 20528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:05.544664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.195:41539)
0926 16:33:05.544968 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:05.544981 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:33:05.545112 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:33:05.545384 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:33:05.545387 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:33:05.545618 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:33:05.545711 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:05.545717 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:05.546554 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:05.546558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:05.547119 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:05.547127 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:05.547588 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:05.547592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:05.547679 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:05.548183 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:33:05.548195 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:33:05.550054 (+ 1859us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":100,"threads_started":1}
W20260926 16:33:05.550372 20526 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:33:05.544742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:41539 (local address 127.11.8.254:36337)
0926 16:33:05.544905 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:05.544911 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:05.545080 (+ 169us) server_negotiation.cc:407] Connection header received
0926 16:33:05.545212 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:05.545216 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:05.545269 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:05.545354 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:33:05.545853 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:05.546441 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:05.547242 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:05.547471 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:05.547584 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:05.550195 (+ 2611us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.195:41539: BlockingRecv error: recv got EOF from 127.11.8.195:41539 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Sep 26 16:33:06 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.193@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:33:06 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.193@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:33:06.213297 20529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:06.207546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.193:40889)
0926 16:33:06.207816 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:06.207831 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:33:06.207930 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:33:06.208192 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:33:06.208194 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:33:06.208406 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:33:06.208501 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:06.208506 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:06.209359 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:06.209363 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:06.210386 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:06.210405 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:06.210838 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:06.210841 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:06.210902 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:06.211410 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:33:06.211421 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:33:06.213188 (+ 1767us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":82,"threads_started":1}
W20260926 16:33:06.213442 20530 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:33:06.207600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:40889 (local address 127.11.8.254:36337)
0926 16:33:06.207846 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:06.207849 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:06.207904 (+ 55us) server_negotiation.cc:407] Connection header received
0926 16:33:06.208037 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:06.208039 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:06.208082 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:06.208158 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:33:06.208600 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:06.209249 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:06.210505 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:06.210733 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:06.210814 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:06.213306 (+ 2492us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.193:40889: BlockingRecv error: recv got EOF from 127.11.8.193:40889 (error 108)
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":65,"threads_started":1}
Sep 26 16:33:06 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.194@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 16:33:06 dist-test-slave-2zvq krb5kdc[19471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.194@KRBTEST.COM for oryx/127.11.8.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 16:33:06.369603 20531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:06.363953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:36337 (local address 127.11.8.194:43987)
0926 16:33:06.364271 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:06.364287 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:33:06.364421 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:33:06.364708 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:33:06.364711 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:33:06.364911 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:33:06.364998 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:06.365004 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:06.365796 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:06.365801 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:06.366455 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:06.366463 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:06.366930 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:06.366933 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:06.367012 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:06.367483 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:33:06.367494 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:33:06.369423 (+ 1929us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.254:36337: Server oryx/127.11.8.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":83,"threads_started":1}
W20260926 16:33:06.369671 20530 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:33:06.364095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:43987 (local address 127.11.8.254:36337)
0926 16:33:06.364248 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:06.364253 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:06.364364 (+ 111us) server_negotiation.cc:407] Connection header received
0926 16:33:06.364547 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:06.364550 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:06.364603 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:06.364686 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:33:06.365144 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:06.365686 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:06.366574 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:06.366819 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:06.366929 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:06.369558 (+ 2629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.194:43987: BlockingRecv error: recv got EOF from 127.11.8.194:43987 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 16:33:06.449524 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 20054
I20260926 16:33:06.455150 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 20206
I20260926 16:33:06.460786 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 20339
I20260926 16:33:06.465793 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 19984
2026-09-26T16:33:06Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17949 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-26T16:33:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:33:06Z Disabled control of system clock
I20260926 16:33:06.495148 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:43157
--webserver_interface=127.11.8.254
--webserver_port=0
--builtin_ntp_servers=127.11.8.212:35011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:43157
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260926 16:33:06.600850 20535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:33:06.601105 20535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:33:06.601140 20535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:33:06.605125 20535 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:33:06.605185 20535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:33:06.605201 20535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:33:06.605217 20535 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:33:06.605232 20535 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:33:06.609824 20535 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:35011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:43157
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:43157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:33:06.610947 20535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:33:06.612267 20535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 16:33:06.620745 20535 server_base.cc:1034] running on GCE node
W20260926 16:33:06.620702 20541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:33:06.620678 20543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:33:06.620693 20540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:33:06.621589 20535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:33:06.622761 20535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:33:06.623963 20535 hybrid_clock.cc:648] HybridClock initialized: now 1790440386623931 us; error 44 us; skew 500 ppm
I20260926 16:33:06.625931 20535 webserver.cc:460] Webserver started at http://127.11.8.254:32769/ using document root <none> and password file <none>
I20260926 16:33:06.626596 20535 fs_manager.cc:359] Metadata directory not provided
I20260926 16:33:06.626662 20535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:33:06.626905 20535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:33:06.628924 20535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "c4e8c54cabe2496bb38e1a7a9fdcfca8"
format_stamp: "Formatted at 2026-09-26 16:33:06 on dist-test-slave-2zvq"
server_key: "c63f4db7a88d5b8c4bc92cda144551a3"
server_key_iv: "6e748024c9a4faa18891117109f9644a"
server_key_version: "encryptionkey@0"
I20260926 16:33:06.629484 20535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "c4e8c54cabe2496bb38e1a7a9fdcfca8"
format_stamp: "Formatted at 2026-09-26 16:33:06 on dist-test-slave-2zvq"
server_key: "c63f4db7a88d5b8c4bc92cda144551a3"
server_key_iv: "6e748024c9a4faa18891117109f9644a"
server_key_version: "encryptionkey@0"
I20260926 16:33:06.633143 20535 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260926 16:33:06.635545 20549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:33:06.636693 20535 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 16:33:06.636818 20535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "c4e8c54cabe2496bb38e1a7a9fdcfca8"
format_stamp: "Formatted at 2026-09-26 16:33:06 on dist-test-slave-2zvq"
server_key: "c63f4db7a88d5b8c4bc92cda144551a3"
server_key_iv: "6e748024c9a4faa18891117109f9644a"
server_key_version: "encryptionkey@0"
I20260926 16:33:06.636921 20535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:33:06.652459 20535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:33:06.653146 20535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:33:06.653344 20535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:33:06.660130 20535 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:43157
I20260926 16:33:06.660171 20601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:43157 every 8 connection(s)
I20260926 16:33:06.661084 20535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:33:06.664255 20602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:33:06.669528 20602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4e8c54cabe2496bb38e1a7a9fdcfca8: Bootstrap starting.
I20260926 16:33:06.670253 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 20535
I20260926 16:33:06.670392 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:33:06.670710 11299 external_mini_cluster.cc:1428] Setting key ec15679d82a771a661e306f03e6f7b89
I20260926 16:33:06.672464 20602 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4e8c54cabe2496bb38e1a7a9fdcfca8: Neither blocks nor log segments found. Creating new log.
I20260926 16:33:06.673190 20602 log.cc:824] T 00000000000000000000000000000000 P c4e8c54cabe2496bb38e1a7a9fdcfca8: Log is configured to *not* fsync() on all Append() calls
I20260926 16:33:06.675174 20602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4e8c54cabe2496bb38e1a7a9fdcfca8: No bootstrap required, opened a new log
I20260926 16:33:06.679455 20602 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4e8c54cabe2496bb38e1a7a9fdcfca8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e8c54cabe2496bb38e1a7a9fdcfca8" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 43157 } }
I20260926 16:33:06.679687 20602 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4e8c54cabe2496bb38e1a7a9fdcfca8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:33:06.679766 20602 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4e8c54cabe2496bb38e1a7a9fdcfca8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4e8c54cabe2496bb38e1a7a9fdcfca8, State: Initialized, Role: FOLLOWER
I20260926 16:33:06.679931 20605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:06.671949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50352 (local address 127.11.8.254:43157)
0926 16:33:06.672584 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:06.672605 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:06.672648 (+ 43us) server_negotiation.cc:407] Connection header received
0926 16:33:06.673244 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:06.673260 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:06.673771 (+ 511us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:06.674088 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:33:06.674471 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:06.675464 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:06.676645 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:06.677098 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:06.677212 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:06.678054 (+ 842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:33:06.678059 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:33:06.678070 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:33:06.678536 (+ 466us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:33:06.678589 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:33:06.678609 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:33:06.678682 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 16:33:06.678866 (+ 184us) server_negotiation.cc:299] Negotiation successful
0926 16:33:06.679142 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":131,"threads_started":1}
I20260926 16:33:06.680271 20602 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4e8c54cabe2496bb38e1a7a9fdcfca8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e8c54cabe2496bb38e1a7a9fdcfca8" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 43157 } }
I20260926 16:33:06.680429 20602 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4e8c54cabe2496bb38e1a7a9fdcfca8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:33:06.680496 20602 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4e8c54cabe2496bb38e1a7a9fdcfca8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:33:06.680586 20602 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4e8c54cabe2496bb38e1a7a9fdcfca8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:33:06.681664 20602 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4e8c54cabe2496bb38e1a7a9fdcfca8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e8c54cabe2496bb38e1a7a9fdcfca8" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 43157 } }
I20260926 16:33:06.682006 20602 leader_election.cc:302] T 00000000000000000000000000000000 P c4e8c54cabe2496bb38e1a7a9fdcfca8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4e8c54cabe2496bb38e1a7a9fdcfca8; no voters:
I20260926 16:33:06.682365 20602 leader_election.cc:288] T 00000000000000000000000000000000 P c4e8c54cabe2496bb38e1a7a9fdcfca8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:33:06.682427 20607 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4e8c54cabe2496bb38e1a7a9fdcfca8 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:33:06.682680 20607 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4e8c54cabe2496bb38e1a7a9fdcfca8 [term 1 LEADER]: Becoming Leader. State: Replica: c4e8c54cabe2496bb38e1a7a9fdcfca8, State: Running, Role: LEADER
I20260926 16:33:06.683014 20607 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4e8c54cabe2496bb38e1a7a9fdcfca8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e8c54cabe2496bb38e1a7a9fdcfca8" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 43157 } }
I20260926 16:33:06.683488 20602 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4e8c54cabe2496bb38e1a7a9fdcfca8 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:33:06.684532 20609 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4e8c54cabe2496bb38e1a7a9fdcfca8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4e8c54cabe2496bb38e1a7a9fdcfca8. Latest consensus state: current_term: 1 leader_uuid: "c4e8c54cabe2496bb38e1a7a9fdcfca8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e8c54cabe2496bb38e1a7a9fdcfca8" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 43157 } } }
I20260926 16:33:06.684650 20609 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4e8c54cabe2496bb38e1a7a9fdcfca8 [sys.catalog]: This master's current role is: LEADER
I20260926 16:33:06.685016 20616 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:33:06.684976 20608 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4e8c54cabe2496bb38e1a7a9fdcfca8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4e8c54cabe2496bb38e1a7a9fdcfca8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e8c54cabe2496bb38e1a7a9fdcfca8" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 43157 } } }
I20260926 16:33:06.685125 20608 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4e8c54cabe2496bb38e1a7a9fdcfca8 [sys.catalog]: This master's current role is: LEADER
I20260926 16:33:06.688256 20616 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:33:06.693413 20616 catalog_manager.cc:1347] Generated new cluster ID: 409eb4389c2b4e6e8bb1c1662577a9dc
I20260926 16:33:06.693471 20616 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:33:06.698971 20616 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:33:06.700130 20616 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:33:06.705103 20616 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4e8c54cabe2496bb38e1a7a9fdcfca8: Generated new TSK 0
I20260926 16:33:06.705773 20616 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:33:06.709620 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.193:0
--local_ip_for_outbound_sockets=127.11.8.193
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:43157
--builtin_ntp_servers=127.11.8.212:35011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260926 16:33:06.830775 20626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:33:06.831091 20626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:33:06.831177 20626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:33:06.837056 20626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:33:06.837134 20626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:33:06.837224 20626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.193
I20260926 16:33:06.842300 20626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:35011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:43157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.193
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:33:06.843353 20626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:33:06.844702 20626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:33:06.854012 20634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:33:06.854071 20626 server_base.cc:1034] running on GCE node
W20260926 16:33:06.853999 20631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:33:06.853977 20632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:33:06.854696 20626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:33:06.855263 20626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:33:06.856442 20626 hybrid_clock.cc:648] HybridClock initialized: now 1790440386856429 us; error 25 us; skew 500 ppm
I20260926 16:33:06.858194 20626 webserver.cc:460] Webserver started at http://127.11.8.193:39341/ using document root <none> and password file <none>
I20260926 16:33:06.858765 20626 fs_manager.cc:359] Metadata directory not provided
I20260926 16:33:06.858867 20626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:33:06.859117 20626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:33:06.861083 20626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/data/instance:
uuid: "173fb7b3a0994ee4a210c91523d1faa0"
format_stamp: "Formatted at 2026-09-26 16:33:06 on dist-test-slave-2zvq"
server_key: "acf83bb9bd3e25e33c9553731e59985c"
server_key_iv: "e600bb2980296349114272aac9e9e517"
server_key_version: "encryptionkey@0"
I20260926 16:33:06.861626 20626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance:
uuid: "173fb7b3a0994ee4a210c91523d1faa0"
format_stamp: "Formatted at 2026-09-26 16:33:06 on dist-test-slave-2zvq"
server_key: "acf83bb9bd3e25e33c9553731e59985c"
server_key_iv: "e600bb2980296349114272aac9e9e517"
server_key_version: "encryptionkey@0"
I20260926 16:33:06.865094 20626 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260926 16:33:06.867571 20640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:33:06.868758 20626 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 16:33:06.868885 20626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/wal
uuid: "173fb7b3a0994ee4a210c91523d1faa0"
format_stamp: "Formatted at 2026-09-26 16:33:06 on dist-test-slave-2zvq"
server_key: "acf83bb9bd3e25e33c9553731e59985c"
server_key_iv: "e600bb2980296349114272aac9e9e517"
server_key_version: "encryptionkey@0"
I20260926 16:33:06.869015 20626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:33:06.889606 20626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:33:06.890347 20626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:33:06.890543 20626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:33:06.891124 20626 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:33:06.892045 20626 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:33:06.892114 20626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:33:06.892184 20626 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:33:06.892234 20626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:33:06.901597 20753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.193:35117 every 8 connection(s)
I20260926 16:33:06.901594 20626 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.193:35117
I20260926 16:33:06.902607 20626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
I20260926 16:33:06.906172 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 20626
I20260926 16:33:06.906303 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance
I20260926 16:33:06.906627 11299 external_mini_cluster.cc:1428] Setting key 86d2119397140fc916bf79593473b276
I20260926 16:33:06.908680 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.194:0
--local_ip_for_outbound_sockets=127.11.8.194
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:43157
--builtin_ntp_servers=127.11.8.212:35011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260926 16:33:06.913184 20605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:06.904452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:47625 (local address 127.11.8.254:43157)
0926 16:33:06.904602 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:06.904606 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:06.905810 (+ 1204us) server_negotiation.cc:407] Connection header received
0926 16:33:06.907626 (+ 1816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:06.907630 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:06.907681 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:06.907767 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:33:06.908900 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:06.909752 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:06.911115 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:06.911360 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:06.911480 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:06.912658 (+ 1178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:33:06.912661 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:33:06.912663 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:33:06.912675 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:33:06.912694 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:33:06.912702 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:33:06.912775 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 16:33:06.912988 (+ 213us) server_negotiation.cc:299] Negotiation successful
0926 16:33:06.913049 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 16:33:06.914002 20756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:06.904935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:43157 (local address 127.11.8.193:47625)
0926 16:33:06.905652 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:06.905692 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 16:33:06.907427 (+ 1735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:33:06.907951 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:33:06.907958 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:33:06.908065 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:33:06.908725 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:06.908746 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:06.909935 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:06.909941 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:06.910989 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:06.910998 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:06.911526 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:06.911531 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:06.911707 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:06.912386 (+ 679us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:33:06.912388 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:33:06.912433 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:33:06.912493 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:33:06.912789 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:33:06.912798 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:33:06.912804 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 16:33:06.912988 (+ 184us) client_negotiation.cc:240] Negotiation successful
0926 16:33:06.913257 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":151,"threads_started":1}
I20260926 16:33:06.915424 20754 heartbeater.cc:347] Connected to a master server at 127.11.8.254:43157
I20260926 16:33:06.915655 20754 heartbeater.cc:464] Registering TS with master...
I20260926 16:33:06.916184 20754 heartbeater.cc:511] Master 127.11.8.254:43157 requested a full tablet report, sending...
I20260926 16:33:06.917699 20566 ts_manager.cc:194] Registered new tserver with Master: 173fb7b3a0994ee4a210c91523d1faa0 (127.11.8.193:35117)
I20260926 16:33:06.918889 20566 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.8.193:47625
W20260926 16:33:07.016031 20757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:33:07.016304 20757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:33:07.016367 20757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:33:07.020522 20757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:33:07.020598 20757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:33:07.020684 20757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.194
I20260926 16:33:07.025300 20757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:35011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:43157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.194
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:33:07.026333 20757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:33:07.027662 20757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 16:33:07.037303 20757 server_base.cc:1034] running on GCE node
W20260926 16:33:07.037338 20763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:33:07.037364 20762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:33:07.037484 20765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:33:07.037900 20757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:33:07.038609 20757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:33:07.039799 20757 hybrid_clock.cc:648] HybridClock initialized: now 1790440387039776 us; error 47 us; skew 500 ppm
I20260926 16:33:07.041805 20757 webserver.cc:460] Webserver started at http://127.11.8.194:44901/ using document root <none> and password file <none>
I20260926 16:33:07.042446 20757 fs_manager.cc:359] Metadata directory not provided
I20260926 16:33:07.042538 20757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:33:07.042764 20757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:33:07.044787 20757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/data/instance:
uuid: "78d4f3f2483b4b5ebda6bfb704b78eed"
format_stamp: "Formatted at 2026-09-26 16:33:07 on dist-test-slave-2zvq"
server_key: "ceb72c21d8d68aa275987e99b859ec9d"
server_key_iv: "abcec8f1a013f53805b8bd01d95b7ee3"
server_key_version: "encryptionkey@0"
I20260926 16:33:07.045359 20757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance:
uuid: "78d4f3f2483b4b5ebda6bfb704b78eed"
format_stamp: "Formatted at 2026-09-26 16:33:07 on dist-test-slave-2zvq"
server_key: "ceb72c21d8d68aa275987e99b859ec9d"
server_key_iv: "abcec8f1a013f53805b8bd01d95b7ee3"
server_key_version: "encryptionkey@0"
I20260926 16:33:07.049090 20757 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260926 16:33:07.051637 20771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:33:07.052855 20757 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 16:33:07.052990 20757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/wal
uuid: "78d4f3f2483b4b5ebda6bfb704b78eed"
format_stamp: "Formatted at 2026-09-26 16:33:07 on dist-test-slave-2zvq"
server_key: "ceb72c21d8d68aa275987e99b859ec9d"
server_key_iv: "abcec8f1a013f53805b8bd01d95b7ee3"
server_key_version: "encryptionkey@0"
I20260926 16:33:07.053100 20757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:33:07.079289 20757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:33:07.079977 20757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:33:07.080173 20757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:33:07.080734 20757 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:33:07.081638 20757 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:33:07.081709 20757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:33:07.081779 20757 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:33:07.081822 20757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:33:07.090921 20757 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.194:33301
I20260926 16:33:07.090948 20884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.194:33301 every 8 connection(s)
I20260926 16:33:07.091776 20757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
I20260926 16:33:07.094797 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 20757
I20260926 16:33:07.094959 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance
I20260926 16:33:07.095293 11299 external_mini_cluster.cc:1428] Setting key e49d060bf2fca0885fb254b39273c6b7
I20260926 16:33:07.097322 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.195:0
--local_ip_for_outbound_sockets=127.11.8.195
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:43157
--builtin_ntp_servers=127.11.8.212:35011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260926 16:33:07.102571 20605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:07.093598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:34019 (local address 127.11.8.254:43157)
0926 16:33:07.093757 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:07.093760 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:07.094965 (+ 1205us) server_negotiation.cc:407] Connection header received
0926 16:33:07.096701 (+ 1736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:07.096705 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:07.096750 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:07.096830 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:33:07.097821 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:07.098700 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:07.100333 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:07.100761 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:07.100897 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:07.101975 (+ 1078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:33:07.101980 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:33:07.101985 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:33:07.102002 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:33:07.102028 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:33:07.102041 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:33:07.102132 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 16:33:07.102292 (+ 160us) server_negotiation.cc:299] Negotiation successful
0926 16:33:07.102366 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 16:33:07.103366 20887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:07.094008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:43157 (local address 127.11.8.194:34019)
0926 16:33:07.094800 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:07.094856 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0926 16:33:07.096473 (+ 1617us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:33:07.097035 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:33:07.097043 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:33:07.097121 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:33:07.097682 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:07.097697 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:07.098851 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:07.098857 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:07.100213 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:07.100222 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:07.100905 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:07.100907 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:07.101068 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:07.101682 (+ 614us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:33:07.101683 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:33:07.101721 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:33:07.101780 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:33:07.102133 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:33:07.102140 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:33:07.102144 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 16:33:07.102238 (+ 94us) client_negotiation.cc:240] Negotiation successful
0926 16:33:07.102513 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":109,"threads_started":1}
I20260926 16:33:07.104521 20885 heartbeater.cc:347] Connected to a master server at 127.11.8.254:43157
I20260926 16:33:07.104750 20885 heartbeater.cc:464] Registering TS with master...
I20260926 16:33:07.105237 20885 heartbeater.cc:511] Master 127.11.8.254:43157 requested a full tablet report, sending...
I20260926 16:33:07.106315 20566 ts_manager.cc:194] Registered new tserver with Master: 78d4f3f2483b4b5ebda6bfb704b78eed (127.11.8.194:33301)
I20260926 16:33:07.107191 20566 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.8.194:34019
W20260926 16:33:07.207449 20888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:33:07.207675 20888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:33:07.207710 20888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:33:07.211601 20888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:33:07.211644 20888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:33:07.211695 20888 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.195
I20260926 16:33:07.216260 20888 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:35011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:43157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.195
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:33:07.217149 20888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:33:07.218433 20888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:33:07.227638 20894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:33:07.227626 20893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:33:07.227634 20896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:33:07.227995 20888 server_base.cc:1034] running on GCE node
I20260926 16:33:07.228312 20888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:33:07.228899 20888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:33:07.230063 20888 hybrid_clock.cc:648] HybridClock initialized: now 1790440387230047 us; error 25 us; skew 500 ppm
I20260926 16:33:07.231782 20888 webserver.cc:460] Webserver started at http://127.11.8.195:41979/ using document root <none> and password file <none>
I20260926 16:33:07.232314 20888 fs_manager.cc:359] Metadata directory not provided
I20260926 16:33:07.232375 20888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:33:07.232612 20888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:33:07.234761 20888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/data/instance:
uuid: "65c6e520d2ca4fb58b04ea068b05479c"
format_stamp: "Formatted at 2026-09-26 16:33:07 on dist-test-slave-2zvq"
server_key: "1f9a8d334a8d5a19320025e82453fcb9"
server_key_iv: "2c502bcbf91f4a6454228bfa31647aff"
server_key_version: "encryptionkey@0"
I20260926 16:33:07.235299 20888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance:
uuid: "65c6e520d2ca4fb58b04ea068b05479c"
format_stamp: "Formatted at 2026-09-26 16:33:07 on dist-test-slave-2zvq"
server_key: "1f9a8d334a8d5a19320025e82453fcb9"
server_key_iv: "2c502bcbf91f4a6454228bfa31647aff"
server_key_version: "encryptionkey@0"
I20260926 16:33:07.238658 20888 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 16:33:07.241027 20902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:33:07.242220 20888 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:33:07.242388 20888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/wal
uuid: "65c6e520d2ca4fb58b04ea068b05479c"
format_stamp: "Formatted at 2026-09-26 16:33:07 on dist-test-slave-2zvq"
server_key: "1f9a8d334a8d5a19320025e82453fcb9"
server_key_iv: "2c502bcbf91f4a6454228bfa31647aff"
server_key_version: "encryptionkey@0"
I20260926 16:33:07.242477 20888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:33:07.254786 20888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:33:07.255363 20888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:33:07.255538 20888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:33:07.256100 20888 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:33:07.256960 20888 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:33:07.257005 20888 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:33:07.257063 20888 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:33:07.257094 20888 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:33:07.265698 20888 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.195:40153
I20260926 16:33:07.265719 21015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.195:40153 every 8 connection(s)
I20260926 16:33:07.266649 20888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
I20260926 16:33:07.273665 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 20888
I20260926 16:33:07.273761 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance
I20260926 16:33:07.274031 11299 external_mini_cluster.cc:1428] Setting key 35b0a71960a77033182a0fc20e79d693
I20260926 16:33:07.276324 20605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:07.268443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:55499 (local address 127.11.8.254:43157)
0926 16:33:07.268579 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:07.268582 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:07.269656 (+ 1074us) server_negotiation.cc:407] Connection header received
0926 16:33:07.271082 (+ 1426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:07.271085 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:07.271133 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:07.271394 (+ 261us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:33:07.272304 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:07.272829 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:07.274115 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:07.274446 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:07.274574 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:07.275828 (+ 1254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:33:07.275831 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:33:07.275833 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:33:07.275844 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:33:07.275864 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:33:07.275872 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:33:07.275967 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 16:33:07.276113 (+ 146us) server_negotiation.cc:299] Negotiation successful
0926 16:33:07.276177 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 16:33:07.277168 21018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:07.268859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:43157 (local address 127.11.8.195:55499)
0926 16:33:07.269503 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:07.269542 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 16:33:07.270858 (+ 1316us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:33:07.271514 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:33:07.271521 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:33:07.271602 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:33:07.272168 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:07.272185 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:07.272960 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:07.272963 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:07.273995 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:07.274004 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:07.274611 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:07.274615 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:07.274843 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:07.275538 (+ 695us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:33:07.275540 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:33:07.275593 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:33:07.275671 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:33:07.275952 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:33:07.275961 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:33:07.275967 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 16:33:07.276102 (+ 135us) client_negotiation.cc:240] Negotiation successful
0926 16:33:07.276371 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":112,"threads_started":1}
I20260926 16:33:07.278227 21016 heartbeater.cc:347] Connected to a master server at 127.11.8.254:43157
I20260926 16:33:07.278486 21016 heartbeater.cc:464] Registering TS with master...
I20260926 16:33:07.279009 21016 heartbeater.cc:511] Master 127.11.8.254:43157 requested a full tablet report, sending...
I20260926 16:33:07.279968 20566 ts_manager.cc:194] Registered new tserver with Master: 65c6e520d2ca4fb58b04ea068b05479c (127.11.8.195:40153)
I20260926 16:33:07.280529 20566 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.8.195:55499
I20260926 16:33:07.287897 11299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 16:33:07.290472 20605 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:33:07.289706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50362 (local address 127.11.8.254:43157)
0926 16:33:07.289814 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:07.289818 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:07.289968 (+ 150us) server_negotiation.cc:407] Connection header received
0926 16:33:07.290061 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:07.290063 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:07.290098 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:07.290158 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:33:07.290385 (+ 227us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50362: BlockingRecv error: recv got EOF from 127.0.0.1:50362 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 16:33:07.290695 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 20626
I20260926 16:33:07.296317 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 20757
I20260926 16:33:07.301783 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 20888
I20260926 16:33:07.306998 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 20535
2026-09-26T16:33:07Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (839 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-26T16:33:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:33:07Z Disabled control of system clock
I20260926 16:33:07.335027 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:41397
--webserver_interface=127.11.8.254
--webserver_port=0
--builtin_ntp_servers=127.11.8.212:37221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:41397
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260926 16:33:07.445271 21027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:33:07.445504 21027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:33:07.445538 21027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:33:07.449445 21027 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:33:07.449493 21027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:33:07.449508 21027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:33:07.449524 21027 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:33:07.449539 21027 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:33:07.454066 21027 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:37221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:41397
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:41397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21027
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:33:07.455031 21027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:33:07.456291 21027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:33:07.464534 21033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:33:07.464597 21027 server_base.cc:1034] running on GCE node
W20260926 16:33:07.464548 21032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:33:07.464550 21035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:33:07.465363 21027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:33:07.466470 21027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:33:07.467631 21027 hybrid_clock.cc:648] HybridClock initialized: now 1790440387467617 us; error 27 us; skew 500 ppm
I20260926 16:33:07.469381 21027 webserver.cc:460] Webserver started at http://127.11.8.254:33505/ using document root <none> and password file <none>
I20260926 16:33:07.469928 21027 fs_manager.cc:359] Metadata directory not provided
I20260926 16:33:07.469991 21027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:33:07.470242 21027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:33:07.472265 21027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "1cecc73a668247abb24dac34722704ef"
format_stamp: "Formatted at 2026-09-26 16:33:07 on dist-test-slave-2zvq"
server_key: "1171548c07622e2ccf5b53317758d6f5"
server_key_iv: "b95ed3bc6530efd1c138f85e999e04d0"
server_key_version: "encryptionkey@0"
I20260926 16:33:07.472817 21027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "1cecc73a668247abb24dac34722704ef"
format_stamp: "Formatted at 2026-09-26 16:33:07 on dist-test-slave-2zvq"
server_key: "1171548c07622e2ccf5b53317758d6f5"
server_key_iv: "b95ed3bc6530efd1c138f85e999e04d0"
server_key_version: "encryptionkey@0"
I20260926 16:33:07.476418 21027 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 16:33:07.478819 21041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:33:07.480062 21027 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:33:07.480207 21027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "1cecc73a668247abb24dac34722704ef"
format_stamp: "Formatted at 2026-09-26 16:33:07 on dist-test-slave-2zvq"
server_key: "1171548c07622e2ccf5b53317758d6f5"
server_key_iv: "b95ed3bc6530efd1c138f85e999e04d0"
server_key_version: "encryptionkey@0"
I20260926 16:33:07.480316 21027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:33:07.484807 21027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:33:07.485385 21027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:33:07.485566 21027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:33:07.492319 21027 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:41397
I20260926 16:33:07.492344 21093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:41397 every 8 connection(s)
I20260926 16:33:07.493196 21027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:33:07.496331 21094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:33:07.500088 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 21027
I20260926 16:33:07.500222 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:33:07.500538 11299 external_mini_cluster.cc:1428] Setting key 3b5b7ea62d480406e571791b5d72fcdf
I20260926 16:33:07.502182 21094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cecc73a668247abb24dac34722704ef: Bootstrap starting.
I20260926 16:33:07.504880 21094 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1cecc73a668247abb24dac34722704ef: Neither blocks nor log segments found. Creating new log.
I20260926 16:33:07.505797 21094 log.cc:824] T 00000000000000000000000000000000 P 1cecc73a668247abb24dac34722704ef: Log is configured to *not* fsync() on all Append() calls
I20260926 16:33:07.506652 21097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:07.502002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46174 (local address 127.11.8.254:41397)
0926 16:33:07.502709 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:07.502728 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:07.502770 (+ 42us) server_negotiation.cc:407] Connection header received
0926 16:33:07.503629 (+ 859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:07.503651 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:07.504076 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:07.504423 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:33:07.505215 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:33:07.505221 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:33:07.505236 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:33:07.505247 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:33:07.505311 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:33:07.505331 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:33:07.505447 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0926 16:33:07.505517 (+ 70us) server_negotiation.cc:299] Negotiation successful
0926 16:33:07.505753 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":105,"threads_started":1}
I20260926 16:33:07.507891 21094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cecc73a668247abb24dac34722704ef: No bootstrap required, opened a new log
I20260926 16:33:07.511382 21094 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1cecc73a668247abb24dac34722704ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cecc73a668247abb24dac34722704ef" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 41397 } }
I20260926 16:33:07.511581 21094 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1cecc73a668247abb24dac34722704ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:33:07.511662 21094 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1cecc73a668247abb24dac34722704ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cecc73a668247abb24dac34722704ef, State: Initialized, Role: FOLLOWER
I20260926 16:33:07.512048 21094 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1cecc73a668247abb24dac34722704ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cecc73a668247abb24dac34722704ef" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 41397 } }
I20260926 16:33:07.512187 21094 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1cecc73a668247abb24dac34722704ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:33:07.512246 21094 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cecc73a668247abb24dac34722704ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:33:07.512341 21094 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1cecc73a668247abb24dac34722704ef [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:33:07.513202 21094 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cecc73a668247abb24dac34722704ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cecc73a668247abb24dac34722704ef" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 41397 } }
I20260926 16:33:07.513583 21094 leader_election.cc:302] T 00000000000000000000000000000000 P 1cecc73a668247abb24dac34722704ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1cecc73a668247abb24dac34722704ef; no voters:
I20260926 16:33:07.513901 21094 leader_election.cc:288] T 00000000000000000000000000000000 P 1cecc73a668247abb24dac34722704ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:33:07.514045 21099 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1cecc73a668247abb24dac34722704ef [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:33:07.514269 21099 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1cecc73a668247abb24dac34722704ef [term 1 LEADER]: Becoming Leader. State: Replica: 1cecc73a668247abb24dac34722704ef, State: Running, Role: LEADER
I20260926 16:33:07.514621 21099 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1cecc73a668247abb24dac34722704ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cecc73a668247abb24dac34722704ef" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 41397 } }
I20260926 16:33:07.515026 21094 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1cecc73a668247abb24dac34722704ef [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:33:07.516090 21101 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cecc73a668247abb24dac34722704ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1cecc73a668247abb24dac34722704ef. Latest consensus state: current_term: 1 leader_uuid: "1cecc73a668247abb24dac34722704ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cecc73a668247abb24dac34722704ef" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 41397 } } }
I20260926 16:33:07.516228 21101 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cecc73a668247abb24dac34722704ef [sys.catalog]: This master's current role is: LEADER
I20260926 16:33:07.516393 21100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cecc73a668247abb24dac34722704ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1cecc73a668247abb24dac34722704ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cecc73a668247abb24dac34722704ef" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 41397 } } }
I20260926 16:33:07.516564 21100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cecc73a668247abb24dac34722704ef [sys.catalog]: This master's current role is: LEADER
I20260926 16:33:07.516870 21110 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:33:07.519945 21110 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:33:07.525012 21110 catalog_manager.cc:1347] Generated new cluster ID: 031aec3311944356b404bb79c65f6667
I20260926 16:33:07.525077 21110 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:33:07.541172 21110 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:33:07.541474 21110 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:33:07.546635 21110 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1cecc73a668247abb24dac34722704ef: Generated new TSK 0
I20260926 16:33:07.547271 21110 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:33:07.551822 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.193:0
--local_ip_for_outbound_sockets=127.11.8.193
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:41397
--builtin_ntp_servers=127.11.8.212:37221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260926 16:33:07.660043 21118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:33:07.660298 21118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:33:07.660336 21118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:33:07.664392 21118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:33:07.664444 21118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:33:07.664500 21118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.193
I20260926 16:33:07.669183 21118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:37221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:41397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21118
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.193
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:33:07.670212 21118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:33:07.671626 21118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:33:07.680903 21123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:33:07.681026 21118 server_base.cc:1034] running on GCE node
W20260926 16:33:07.680927 21124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:33:07.680953 21126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:33:07.681536 21118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:33:07.682128 21118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:33:07.683314 21118 hybrid_clock.cc:648] HybridClock initialized: now 1790440387683306 us; error 28 us; skew 500 ppm
I20260926 16:33:07.685201 21118 webserver.cc:460] Webserver started at http://127.11.8.193:45677/ using document root <none> and password file <none>
I20260926 16:33:07.685775 21118 fs_manager.cc:359] Metadata directory not provided
I20260926 16:33:07.685866 21118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:33:07.686095 21118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:33:07.688131 21118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/data/instance:
uuid: "dc70c929d111401ba5cc285f79dede93"
format_stamp: "Formatted at 2026-09-26 16:33:07 on dist-test-slave-2zvq"
server_key: "32c6c0b82b72da28caa06d841ee97913"
server_key_iv: "59b46228a27b8c4474e2e9b716c77a0e"
server_key_version: "encryptionkey@0"
I20260926 16:33:07.688699 21118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance:
uuid: "dc70c929d111401ba5cc285f79dede93"
format_stamp: "Formatted at 2026-09-26 16:33:07 on dist-test-slave-2zvq"
server_key: "32c6c0b82b72da28caa06d841ee97913"
server_key_iv: "59b46228a27b8c4474e2e9b716c77a0e"
server_key_version: "encryptionkey@0"
I20260926 16:33:07.692206 21118 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260926 16:33:07.694613 21132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:33:07.695752 21118 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:33:07.695881 21118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/wal
uuid: "dc70c929d111401ba5cc285f79dede93"
format_stamp: "Formatted at 2026-09-26 16:33:07 on dist-test-slave-2zvq"
server_key: "32c6c0b82b72da28caa06d841ee97913"
server_key_iv: "59b46228a27b8c4474e2e9b716c77a0e"
server_key_version: "encryptionkey@0"
I20260926 16:33:07.695996 21118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:33:07.700359 21118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:33:07.700981 21118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:33:07.701179 21118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:33:07.701699 21118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:33:07.702605 21118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:33:07.702682 21118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:33:07.702750 21118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:33:07.702786 21118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:33:07.711158 21118 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.193:44981
I20260926 16:33:07.711195 21245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.193:44981 every 8 connection(s)
I20260926 16:33:07.711982 21118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
I20260926 16:33:07.717650 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 21118
I20260926 16:33:07.717774 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance
I20260926 16:33:07.717928 21097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:07.713680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:48085 (local address 127.11.8.254:41397)
0926 16:33:07.713813 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:07.713816 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:07.714930 (+ 1114us) server_negotiation.cc:407] Connection header received
0926 16:33:07.716254 (+ 1324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:07.716257 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:07.716292 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:07.716363 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:33:07.717432 (+ 1069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:33:07.717436 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:33:07.717438 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:33:07.717449 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:33:07.717468 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:33:07.717476 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:33:07.717565 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 16:33:07.717750 (+ 185us) server_negotiation.cc:299] Negotiation successful
0926 16:33:07.717796 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 16:33:07.718128 11299 external_mini_cluster.cc:1428] Setting key 18ecea920158f002e08a47ae34c35339
I20260926 16:33:07.718945 21248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:07.714041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:41397 (local address 127.11.8.193:48085)
0926 16:33:07.714763 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:07.714802 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 16:33:07.716074 (+ 1272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:33:07.716489 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:33:07.716495 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:33:07.716577 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:33:07.717171 (+ 594us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:33:07.717173 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:33:07.717210 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:33:07.717276 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:33:07.717570 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:33:07.717579 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:33:07.717586 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 16:33:07.717773 (+ 187us) client_negotiation.cc:240] Negotiation successful
0926 16:33:07.718011 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":106,"threads_started":1}
I20260926 16:33:07.720100 21246 heartbeater.cc:347] Connected to a master server at 127.11.8.254:41397
I20260926 16:33:07.720166 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.194:0
--local_ip_for_outbound_sockets=127.11.8.194
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:41397
--builtin_ntp_servers=127.11.8.212:37221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260926 16:33:07.720325 21246 heartbeater.cc:464] Registering TS with master...
I20260926 16:33:07.720757 21246 heartbeater.cc:511] Master 127.11.8.254:41397 requested a full tablet report, sending...
I20260926 16:33:07.722180 21058 ts_manager.cc:194] Registered new tserver with Master: dc70c929d111401ba5cc285f79dede93 (127.11.8.193:44981)
W20260926 16:33:07.827453 21249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:33:07.827696 21249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:33:07.827734 21249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:33:07.831581 21249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:33:07.831624 21249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:33:07.831677 21249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.194
I20260926 16:33:07.836153 21249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:37221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:41397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.194
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:33:07.837062 21249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:33:07.838337 21249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:33:07.847842 21257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:33:07.847823 21255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:33:07.847815 21254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:33:07.848183 21249 server_base.cc:1034] running on GCE node
I20260926 16:33:07.848632 21249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:33:07.849232 21249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:33:07.850410 21249 hybrid_clock.cc:648] HybridClock initialized: now 1790440387850392 us; error 29 us; skew 500 ppm
I20260926 16:33:07.852363 21249 webserver.cc:460] Webserver started at http://127.11.8.194:32811/ using document root <none> and password file <none>
I20260926 16:33:07.852941 21249 fs_manager.cc:359] Metadata directory not provided
I20260926 16:33:07.853032 21249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:33:07.853238 21249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:33:07.855298 21249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/data/instance:
uuid: "92c0b1c42d0041e9bc59db3a589713d7"
format_stamp: "Formatted at 2026-09-26 16:33:07 on dist-test-slave-2zvq"
server_key: "6036a416333fecf9fa363f2891221e1c"
server_key_iv: "4d1740cd88cfe4c8304e9b0ea66a4b57"
server_key_version: "encryptionkey@0"
I20260926 16:33:07.855862 21249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance:
uuid: "92c0b1c42d0041e9bc59db3a589713d7"
format_stamp: "Formatted at 2026-09-26 16:33:07 on dist-test-slave-2zvq"
server_key: "6036a416333fecf9fa363f2891221e1c"
server_key_iv: "4d1740cd88cfe4c8304e9b0ea66a4b57"
server_key_version: "encryptionkey@0"
I20260926 16:33:07.859437 21249 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260926 16:33:07.862103 21263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:33:07.863440 21249 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260926 16:33:07.863586 21249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/wal
uuid: "92c0b1c42d0041e9bc59db3a589713d7"
format_stamp: "Formatted at 2026-09-26 16:33:07 on dist-test-slave-2zvq"
server_key: "6036a416333fecf9fa363f2891221e1c"
server_key_iv: "4d1740cd88cfe4c8304e9b0ea66a4b57"
server_key_version: "encryptionkey@0"
I20260926 16:33:07.863703 21249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:33:07.868196 21249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:33:07.868788 21249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:33:07.868963 21249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:33:07.869486 21249 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:33:07.870428 21249 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:33:07.870492 21249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:33:07.870559 21249 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:33:07.870609 21249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:33:07.879465 21249 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.194:40779
I20260926 16:33:07.879488 21376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.194:40779 every 8 connection(s)
I20260926 16:33:07.880326 21249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
I20260926 16:33:07.886452 21097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:07.882160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:33113 (local address 127.11.8.254:41397)
0926 16:33:07.882339 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:07.882344 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:07.883409 (+ 1065us) server_negotiation.cc:407] Connection header received
0926 16:33:07.884658 (+ 1249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:07.884660 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:07.884698 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:07.884774 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:33:07.885912 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:33:07.885915 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:33:07.885917 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:33:07.885928 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:33:07.885945 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:33:07.885953 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:33:07.886040 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 16:33:07.886198 (+ 158us) server_negotiation.cc:299] Negotiation successful
0926 16:33:07.886247 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 16:33:07.886437 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 21249
I20260926 16:33:07.886556 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance
I20260926 16:33:07.886857 11299 external_mini_cluster.cc:1428] Setting key 4a1c8e3c1915c6d3d01c1502bb083436
I20260926 16:33:07.887302 21379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:07.882623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:41397 (local address 127.11.8.194:33113)
0926 16:33:07.883253 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:07.883313 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0926 16:33:07.884465 (+ 1152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:33:07.884921 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:33:07.884928 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:33:07.885032 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:33:07.885656 (+ 624us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:33:07.885658 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:33:07.885697 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:33:07.885764 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:33:07.886030 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:33:07.886034 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:33:07.886038 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 16:33:07.886157 (+ 119us) client_negotiation.cc:240] Negotiation successful
0926 16:33:07.886391 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":106,"threads_started":1}
I20260926 16:33:07.888793 21377 heartbeater.cc:347] Connected to a master server at 127.11.8.254:41397
I20260926 16:33:07.889066 21377 heartbeater.cc:464] Registering TS with master...
I20260926 16:33:07.889045 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.195:0
--local_ip_for_outbound_sockets=127.11.8.195
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:41397
--builtin_ntp_servers=127.11.8.212:37221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260926 16:33:07.889519 21377 heartbeater.cc:511] Master 127.11.8.254:41397 requested a full tablet report, sending...
I20260926 16:33:07.890688 21057 ts_manager.cc:194] Registered new tserver with Master: 92c0b1c42d0041e9bc59db3a589713d7 (127.11.8.194:40779)
W20260926 16:33:07.997750 21380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:33:07.997995 21380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:33:07.998032 21380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:33:08.002363 21380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:33:08.002413 21380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:33:08.002504 21380 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.195
I20260926 16:33:08.007184 21380 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:37221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:41397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.195
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:33:08.008176 21380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:33:08.009495 21380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:33:08.018913 21386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:33:08.019084 21380 server_base.cc:1034] running on GCE node
W20260926 16:33:08.018923 21388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:33:08.019232 21385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:33:08.019604 21380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:33:08.020265 21380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:33:08.021445 21380 hybrid_clock.cc:648] HybridClock initialized: now 1790440388021425 us; error 31 us; skew 500 ppm
I20260926 16:33:08.023294 21380 webserver.cc:460] Webserver started at http://127.11.8.195:43453/ using document root <none> and password file <none>
I20260926 16:33:08.023849 21380 fs_manager.cc:359] Metadata directory not provided
I20260926 16:33:08.023938 21380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:33:08.024163 21380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:33:08.026117 21380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/data/instance:
uuid: "affc3db3d3784c9990226c2334612ce9"
format_stamp: "Formatted at 2026-09-26 16:33:08 on dist-test-slave-2zvq"
server_key: "cc3195e6abf31c6721fe819ef14f2254"
server_key_iv: "b7268846299d9aef55d607a6e38ccb20"
server_key_version: "encryptionkey@0"
I20260926 16:33:08.026718 21380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance:
uuid: "affc3db3d3784c9990226c2334612ce9"
format_stamp: "Formatted at 2026-09-26 16:33:08 on dist-test-slave-2zvq"
server_key: "cc3195e6abf31c6721fe819ef14f2254"
server_key_iv: "b7268846299d9aef55d607a6e38ccb20"
server_key_version: "encryptionkey@0"
I20260926 16:33:08.030211 21380 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260926 16:33:08.032594 21394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:33:08.033813 21380 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260926 16:33:08.033943 21380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/wal
uuid: "affc3db3d3784c9990226c2334612ce9"
format_stamp: "Formatted at 2026-09-26 16:33:08 on dist-test-slave-2zvq"
server_key: "cc3195e6abf31c6721fe819ef14f2254"
server_key_iv: "b7268846299d9aef55d607a6e38ccb20"
server_key_version: "encryptionkey@0"
I20260926 16:33:08.034066 21380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:33:08.038347 21380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:33:08.038921 21380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:33:08.039100 21380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:33:08.039608 21380 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:33:08.040441 21380 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:33:08.040508 21380 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:33:08.040558 21380 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:33:08.040606 21380 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:33:08.048887 21380 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.195:45417
I20260926 16:33:08.048938 21507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.195:45417 every 8 connection(s)
I20260926 16:33:08.049739 21380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
I20260926 16:33:08.055368 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 21380
I20260926 16:33:08.055457 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance
I20260926 16:33:08.055748 11299 external_mini_cluster.cc:1428] Setting key e61bbfcc81d9364d0bd4abb4db65087e
I20260926 16:33:08.056010 21097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:08.051487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:42893 (local address 127.11.8.254:41397)
0926 16:33:08.051646 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:08.051650 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:08.052721 (+ 1071us) server_negotiation.cc:407] Connection header received
0926 16:33:08.054243 (+ 1522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:08.054246 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:08.054312 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:08.054443 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:33:08.055547 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:33:08.055550 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:33:08.055553 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:33:08.055569 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:33:08.055591 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:33:08.055603 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:33:08.055685 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 16:33:08.055833 (+ 148us) server_negotiation.cc:299] Negotiation successful
0926 16:33:08.055877 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 16:33:08.056929 21510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:08.051862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:41397 (local address 127.11.8.195:42893)
0926 16:33:08.052554 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:08.052592 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 16:33:08.054011 (+ 1419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:33:08.054601 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:33:08.054608 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:33:08.054694 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:33:08.055273 (+ 579us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:33:08.055275 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:33:08.055313 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:33:08.055377 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:33:08.055678 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:33:08.055686 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:33:08.055694 (+ 8us) client_negotiation.cc:769] Sending connection context
0926 16:33:08.055863 (+ 169us) client_negotiation.cc:240] Negotiation successful
0926 16:33:08.056121 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":181,"threads_started":1}
I20260926 16:33:08.058040 21508 heartbeater.cc:347] Connected to a master server at 127.11.8.254:41397
I20260926 16:33:08.058315 21508 heartbeater.cc:464] Registering TS with master...
I20260926 16:33:08.058722 21508 heartbeater.cc:511] Master 127.11.8.254:41397 requested a full tablet report, sending...
I20260926 16:33:08.059726 21057 ts_manager.cc:194] Registered new tserver with Master: affc3db3d3784c9990226c2334612ce9 (127.11.8.195:45417)
I20260926 16:33:08.069284 11299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 16:33:08.071905 21097 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:33:08.071135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46186 (local address 127.11.8.254:41397)
0926 16:33:08.071274 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:08.071281 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:08.071385 (+ 104us) server_negotiation.cc:407] Connection header received
0926 16:33:08.071477 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:08.071479 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:08.071510 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:08.071570 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:33:08.071781 (+ 211us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46186: BlockingRecv error: recv got EOF from 127.0.0.1:46186 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 16:33:08.072021 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 21118
I20260926 16:33:08.077705 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 21249
I20260926 16:33:08.083344 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 21380
I20260926 16:33:08.088186 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 21027
2026-09-26T16:33:08Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (780 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:33:08 dist-test-slave-2zvq krb5kdc[21518](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:33:08 dist-test-slave-2zvq krb5kdc[21518](info): set up 2 sockets
Sep 26 16:33:08 dist-test-slave-2zvq krb5kdc[21518](info): commencing operation
krb5kdc: starting...
W20260926 16:33:09.137070 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:33:09 dist-test-slave-2zvq krb5kdc[21518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440389, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:33:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:33:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
I20260926 16:33:09.253445 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:38593
--webserver_interface=127.11.8.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:37235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:38593
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:33:09.359843 21531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:33:09.360083 21531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:33:09.360121 21531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:33:09.364091 21531 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:33:09.364138 21531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:33:09.364157 21531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:33:09.364173 21531 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:33:09.364213 21531 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:33:09.369202 21531 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:37235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:38593
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:38593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:33:09.370138 21531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:33:09.371091 21531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:33:09.379735 21537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:33:09.379741 21539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:33:09.379788 21536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:33:09.379944 21531 server_base.cc:1034] running on GCE node
I20260926 16:33:09.380558 21531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:33:09.381651 21531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:33:09.382838 21531 hybrid_clock.cc:648] HybridClock initialized: now 1790440389382828 us; error 29 us; skew 500 ppm
Sep 26 16:33:09 dist-test-slave-2zvq krb5kdc[21518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440389, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:33:09.385636 21531 init.cc:377] Logged in from keytab as kudu/127.11.8.254@KRBTEST.COM (short username kudu)
I20260926 16:33:09.386497 21531 webserver.cc:460] Webserver started at http://127.11.8.254:37639/ using document root <none> and password file <none>
I20260926 16:33:09.386988 21531 fs_manager.cc:359] Metadata directory not provided
I20260926 16:33:09.387045 21531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:33:09.387199 21531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:33:09.389221 21531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "34789ec4754d444caecc6628ad67452d"
format_stamp: "Formatted at 2026-09-26 16:33:09 on dist-test-slave-2zvq"
server_key: "15c5625d1d836062f6bffbdeee20859b"
server_key_iv: "99f3f35347c30b76e48cc9f08ad45e89"
server_key_version: "encryptionkey@0"
I20260926 16:33:09.389719 21531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "34789ec4754d444caecc6628ad67452d"
format_stamp: "Formatted at 2026-09-26 16:33:09 on dist-test-slave-2zvq"
server_key: "15c5625d1d836062f6bffbdeee20859b"
server_key_iv: "99f3f35347c30b76e48cc9f08ad45e89"
server_key_version: "encryptionkey@0"
I20260926 16:33:09.393195 21531 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260926 16:33:09.395596 21546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:33:09.396867 21531 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 16:33:09.396971 21531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "34789ec4754d444caecc6628ad67452d"
format_stamp: "Formatted at 2026-09-26 16:33:09 on dist-test-slave-2zvq"
server_key: "15c5625d1d836062f6bffbdeee20859b"
server_key_iv: "99f3f35347c30b76e48cc9f08ad45e89"
server_key_version: "encryptionkey@0"
I20260926 16:33:09.397045 21531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:33:09.432754 21531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:33:09.436067 21531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:33:09.436218 21531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:33:09.443043 21531 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:38593
I20260926 16:33:09.443328 21598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:38593 every 8 connection(s)
I20260926 16:33:09.444028 21531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:33:09.446671 21599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:33:09.449257 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 21531
I20260926 16:33:09.449352 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:33:09.449636 11299 external_mini_cluster.cc:1428] Setting key 3fef487737a94a48dc95d1f4c40aafb1
I20260926 16:33:09.452636 21599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34789ec4754d444caecc6628ad67452d: Bootstrap starting.
I20260926 16:33:09.455322 21599 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34789ec4754d444caecc6628ad67452d: Neither blocks nor log segments found. Creating new log.
I20260926 16:33:09.456056 21599 log.cc:824] T 00000000000000000000000000000000 P 34789ec4754d444caecc6628ad67452d: Log is configured to *not* fsync() on all Append() calls
I20260926 16:33:09.457921 21599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34789ec4754d444caecc6628ad67452d: No bootstrap required, opened a new log
Sep 26 16:33:09 dist-test-slave-2zvq krb5kdc[21518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440389, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:33:09.461826 21599 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34789ec4754d444caecc6628ad67452d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34789ec4754d444caecc6628ad67452d" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 38593 } }
I20260926 16:33:09.462102 21599 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34789ec4754d444caecc6628ad67452d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:33:09.462176 21599 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34789ec4754d444caecc6628ad67452d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34789ec4754d444caecc6628ad67452d, State: Initialized, Role: FOLLOWER
I20260926 16:33:09.462658 21599 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34789ec4754d444caecc6628ad67452d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34789ec4754d444caecc6628ad67452d" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 38593 } }
I20260926 16:33:09.462826 21599 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34789ec4754d444caecc6628ad67452d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:33:09.462896 21599 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34789ec4754d444caecc6628ad67452d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:33:09.462990 21599 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34789ec4754d444caecc6628ad67452d [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:33:09.464069 21599 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34789ec4754d444caecc6628ad67452d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34789ec4754d444caecc6628ad67452d" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 38593 } }
I20260926 16:33:09.464447 21599 leader_election.cc:302] T 00000000000000000000000000000000 P 34789ec4754d444caecc6628ad67452d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34789ec4754d444caecc6628ad67452d; no voters:
I20260926 16:33:09.464758 21599 leader_election.cc:288] T 00000000000000000000000000000000 P 34789ec4754d444caecc6628ad67452d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:33:09.464926 21604 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34789ec4754d444caecc6628ad67452d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:33:09.465198 21604 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34789ec4754d444caecc6628ad67452d [term 1 LEADER]: Becoming Leader. State: Replica: 34789ec4754d444caecc6628ad67452d, State: Running, Role: LEADER
I20260926 16:33:09.465404 21602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:09.451096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44982 (local address 127.11.8.254:38593)
0926 16:33:09.451510 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:09.451540 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:09.451583 (+ 43us) server_negotiation.cc:407] Connection header received
0926 16:33:09.452348 (+ 765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:09.452371 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:09.452784 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:09.453134 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:33:09.453765 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:09.455254 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:09.456149 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:09.456719 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:09.456907 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:09.459402 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:33:09.459408 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:33:09.459423 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:33:09.459456 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:33:09.462013 (+ 2557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:33:09.462561 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:33:09.462566 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:33:09.462571 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:33:09.462711 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:33:09.463165 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:33:09.463169 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:33:09.463170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:33:09.463660 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:33:09.463895 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0926 16:33:09.464067 (+ 172us) server_negotiation.cc:299] Negotiation successful
0926 16:33:09.464411 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":108,"threads_started":1}
I20260926 16:33:09.465515 21604 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34789ec4754d444caecc6628ad67452d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34789ec4754d444caecc6628ad67452d" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 38593 } }
I20260926 16:33:09.465975 21599 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34789ec4754d444caecc6628ad67452d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:33:09.466709 21604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34789ec4754d444caecc6628ad67452d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34789ec4754d444caecc6628ad67452d. Latest consensus state: current_term: 1 leader_uuid: "34789ec4754d444caecc6628ad67452d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34789ec4754d444caecc6628ad67452d" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 38593 } } }
I20260926 16:33:09.466818 21604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34789ec4754d444caecc6628ad67452d [sys.catalog]: This master's current role is: LEADER
I20260926 16:33:09.467546 21605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34789ec4754d444caecc6628ad67452d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34789ec4754d444caecc6628ad67452d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34789ec4754d444caecc6628ad67452d" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 38593 } } }
I20260926 16:33:09.467650 21605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34789ec4754d444caecc6628ad67452d [sys.catalog]: This master's current role is: LEADER
I20260926 16:33:09.467705 21612 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:33:09.471205 21612 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:33:09.476815 21612 catalog_manager.cc:1347] Generated new cluster ID: 168ca174df8349ec9dc3e67109c0fc6d
I20260926 16:33:09.476897 21612 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:33:09.488808 21612 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:33:09.489703 21612 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:33:09.501137 21612 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34789ec4754d444caecc6628ad67452d: Generated new TSK 0
I20260926 16:33:09.501943 21612 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.8.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab.
Entry for principal kudu/127.11.8.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab.
I20260926 16:33:09.527396 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.193:0
--local_ip_for_outbound_sockets=127.11.8.193
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:38593
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:37235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:33:09.635838 21625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:33:09.636133 21625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:33:09.636211 21625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:33:09.640275 21625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:33:09.640363 21625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:33:09.640457 21625 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.193
I20260926 16:33:09.645618 21625 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:37235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:38593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21625
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.193
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:33:09.646733 21625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:33:09.647770 21625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:33:09.657435 21630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:33:09.657619 21631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:33:09.657501 21625 server_base.cc:1034] running on GCE node
W20260926 16:33:09.657435 21633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:33:09.658109 21625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:33:09.658805 21625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:33:09.660059 21625 hybrid_clock.cc:648] HybridClock initialized: now 1790440389660027 us; error 43 us; skew 500 ppm
Sep 26 16:33:09 dist-test-slave-2zvq krb5kdc[21518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440389, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:33:09.662868 21625 init.cc:377] Logged in from keytab as kudu/127.11.8.193@KRBTEST.COM (short username kudu)
I20260926 16:33:09.663712 21625 webserver.cc:460] Webserver started at http://127.11.8.193:34607/ using document root <none> and password file <none>
I20260926 16:33:09.664269 21625 fs_manager.cc:359] Metadata directory not provided
I20260926 16:33:09.664330 21625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:33:09.664539 21625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:33:09.666764 21625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/data/instance:
uuid: "ebc9a5c407fe44e3bd281fb34bb3d251"
format_stamp: "Formatted at 2026-09-26 16:33:09 on dist-test-slave-2zvq"
server_key: "2403119f72c69536fa5fc6b8b955eba3"
server_key_iv: "57ef5f7942c24060bf6a919e9bf4141e"
server_key_version: "encryptionkey@0"
I20260926 16:33:09.667445 21625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance:
uuid: "ebc9a5c407fe44e3bd281fb34bb3d251"
format_stamp: "Formatted at 2026-09-26 16:33:09 on dist-test-slave-2zvq"
server_key: "2403119f72c69536fa5fc6b8b955eba3"
server_key_iv: "57ef5f7942c24060bf6a919e9bf4141e"
server_key_version: "encryptionkey@0"
I20260926 16:33:09.671120 21625 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:33:09.673425 21640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:33:09.674646 21625 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:33:09.674774 21625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/wal
uuid: "ebc9a5c407fe44e3bd281fb34bb3d251"
format_stamp: "Formatted at 2026-09-26 16:33:09 on dist-test-slave-2zvq"
server_key: "2403119f72c69536fa5fc6b8b955eba3"
server_key_iv: "57ef5f7942c24060bf6a919e9bf4141e"
server_key_version: "encryptionkey@0"
I20260926 16:33:09.674880 21625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:33:09.685930 21625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:33:09.688984 21625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:33:09.689191 21625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:33:09.689751 21625 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:33:09.690558 21625 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:33:09.690634 21625 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:33:09.690701 21625 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:33:09.690748 21625 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:33:09.699492 21625 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.193:40121
I20260926 16:33:09.699520 21753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.193:40121 every 8 connection(s)
I20260926 16:33:09.700392 21625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
I20260926 16:33:09.703383 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 21625
I20260926 16:33:09.703511 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance
I20260926 16:33:09.703794 11299 external_mini_cluster.cc:1428] Setting key 0e293bb558ecbf1cd075ec92937fc189
Sep 26 16:33:09 dist-test-slave-2zvq krb5kdc[21518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440389, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.8.194@KRBTEST.COM; defaulting to no policy
I20260926 16:33:09.715404 21602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:09.702145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:60237 (local address 127.11.8.254:38593)
0926 16:33:09.702319 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:09.702323 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:09.703103 (+ 780us) server_negotiation.cc:407] Connection header received
0926 16:33:09.704643 (+ 1540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:09.704646 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:09.704689 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:09.704763 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:33:09.706055 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:09.706957 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:09.708632 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:09.708876 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:09.708988 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:09.711716 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:33:09.711720 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:33:09.711722 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:33:09.711744 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:33:09.713176 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:33:09.713755 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:33:09.713763 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:33:09.713764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:33:09.713837 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:33:09.714172 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:33:09.714177 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:33:09.714179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:33:09.714452 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:33:09.714561 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 16:33:09.715055 (+ 494us) server_negotiation.cc:299] Negotiation successful
0926 16:33:09.715240 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.194@KRBTEST.COM" created.
I20260926 16:33:09.716229 21756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:09.702554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:38593 (local address 127.11.8.193:60237)
0926 16:33:09.702964 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:09.703004 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 16:33:09.704409 (+ 1405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:33:09.704953 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:33:09.704963 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:33:09.705444 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:33:09.705904 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:09.705917 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:09.707102 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:09.707108 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:09.708487 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:09.708499 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:09.709028 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:09.709033 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:09.709211 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:09.709825 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:33:09.709851 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:33:09.711549 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:33:09.713305 (+ 1756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:33:09.713310 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:33:09.713324 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:33:09.713642 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:33:09.713928 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:33:09.713932 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:33:09.713934 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:33:09.714077 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:33:09.714562 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:33:09.714570 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:33:09.714807 (+ 237us) client_negotiation.cc:769] Sending connection context
0926 16:33:09.714992 (+ 185us) client_negotiation.cc:240] Negotiation successful
0926 16:33:09.715321 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":108,"threads_started":1}
I20260926 16:33:09.717564 21754 heartbeater.cc:347] Connected to a master server at 127.11.8.254:38593
I20260926 16:33:09.717796 21754 heartbeater.cc:464] Registering TS with master...
I20260926 16:33:09.718320 21754 heartbeater.cc:511] Master 127.11.8.254:38593 requested a full tablet report, sending...
I20260926 16:33:09.719756 21563 ts_manager.cc:194] Registered new tserver with Master: ebc9a5c407fe44e3bd281fb34bb3d251 (127.11.8.193:40121)
I20260926 16:33:09.720845 21563 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.193@KRBTEST.COM'} at 127.11.8.193:60237
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab.
Entry for principal kudu/127.11.8.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab.
I20260926 16:33:09.729223 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.194:0
--local_ip_for_outbound_sockets=127.11.8.194
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:38593
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:37235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:33:09.834905 21759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:33:09.835135 21759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:33:09.835175 21759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:33:09.839015 21759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:33:09.839066 21759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:33:09.839120 21759 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.194
I20260926 16:33:09.844090 21759 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:37235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:38593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.194
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:33:09.845023 21759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:33:09.845959 21759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:33:09.855150 21764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:33:09.855234 21765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:33:09.855150 21767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:33:09.855412 21759 server_base.cc:1034] running on GCE node
I20260926 16:33:09.855904 21759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:33:09.856513 21759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:33:09.857705 21759 hybrid_clock.cc:648] HybridClock initialized: now 1790440389857675 us; error 53 us; skew 500 ppm
Sep 26 16:33:09 dist-test-slave-2zvq krb5kdc[21518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440389, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:33:09.860534 21759 init.cc:377] Logged in from keytab as kudu/127.11.8.194@KRBTEST.COM (short username kudu)
I20260926 16:33:09.861320 21759 webserver.cc:460] Webserver started at http://127.11.8.194:42497/ using document root <none> and password file <none>
I20260926 16:33:09.861824 21759 fs_manager.cc:359] Metadata directory not provided
I20260926 16:33:09.861879 21759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:33:09.862038 21759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:33:09.864099 21759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/data/instance:
uuid: "b746c882da704aba801a1b1eb5982c97"
format_stamp: "Formatted at 2026-09-26 16:33:09 on dist-test-slave-2zvq"
server_key: "96088b906a13e936125aa7098b102480"
server_key_iv: "b67421927dc8135fbb4be51ffbe17494"
server_key_version: "encryptionkey@0"
I20260926 16:33:09.864621 21759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance:
uuid: "b746c882da704aba801a1b1eb5982c97"
format_stamp: "Formatted at 2026-09-26 16:33:09 on dist-test-slave-2zvq"
server_key: "96088b906a13e936125aa7098b102480"
server_key_iv: "b67421927dc8135fbb4be51ffbe17494"
server_key_version: "encryptionkey@0"
I20260926 16:33:09.868597 21759 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 16:33:09.870903 21774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:33:09.872092 21759 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:33:09.872185 21759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/wal
uuid: "b746c882da704aba801a1b1eb5982c97"
format_stamp: "Formatted at 2026-09-26 16:33:09 on dist-test-slave-2zvq"
server_key: "96088b906a13e936125aa7098b102480"
server_key_iv: "b67421927dc8135fbb4be51ffbe17494"
server_key_version: "encryptionkey@0"
I20260926 16:33:09.872265 21759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:33:09.894554 21759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:33:09.897617 21759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:33:09.897768 21759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:33:09.898298 21759 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:33:09.899174 21759 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:33:09.899221 21759 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:33:09.899261 21759 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:33:09.899277 21759 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:33:09.909206 21759 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.194:45757
I20260926 16:33:09.909232 21887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.194:45757 every 8 connection(s)
I20260926 16:33:09.910068 21759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
I20260926 16:33:09.914873 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 21759
I20260926 16:33:09.915002 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance
I20260926 16:33:09.915311 11299 external_mini_cluster.cc:1428] Setting key bc22a1ba4039c31c38708d23a13a0eaa
Sep 26 16:33:09 dist-test-slave-2zvq krb5kdc[21518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440389, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:33:09.924705 21602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:09.911901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:56367 (local address 127.11.8.254:38593)
0926 16:33:09.912076 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:09.912082 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:09.912837 (+ 755us) server_negotiation.cc:407] Connection header received
0926 16:33:09.914199 (+ 1362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:09.914202 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:09.914250 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:09.914387 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:33:09.915995 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:09.916525 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:09.917677 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:09.917901 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:09.918010 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:09.920809 (+ 2799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:33:09.920815 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:33:09.920820 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:33:09.920895 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:33:09.922653 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:33:09.923138 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:33:09.923143 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:33:09.923146 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:33:09.923245 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:33:09.923557 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:33:09.923561 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:33:09.923563 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:33:09.923802 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:33:09.923921 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0926 16:33:09.924375 (+ 454us) server_negotiation.cc:299] Negotiation successful
0926 16:33:09.924508 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
WARNING: no policy specified for kudu/127.11.8.195@KRBTEST.COM; defaulting to no policy
I20260926 16:33:09.925441 21890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:09.912223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:38593 (local address 127.11.8.194:56367)
0926 16:33:09.912682 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:09.912723 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 16:33:09.913990 (+ 1267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:33:09.914536 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:33:09.914543 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:33:09.915144 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:33:09.915792 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:09.915861 (+ 69us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:09.916646 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:09.916650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:09.917543 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:09.917551 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:09.918046 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:09.918050 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:09.918304 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:09.918940 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:33:09.918965 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:33:09.920611 (+ 1646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:33:09.922768 (+ 2157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:33:09.922772 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:33:09.922783 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:33:09.923034 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:33:09.923338 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:33:09.923340 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:33:09.923341 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:33:09.923463 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:33:09.923911 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:33:09.923916 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:33:09.924134 (+ 218us) client_negotiation.cc:769] Sending connection context
0926 16:33:09.924284 (+ 150us) client_negotiation.cc:240] Negotiation successful
0926 16:33:09.924531 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":111,"threads_started":1}
I20260926 16:33:09.926707 21888 heartbeater.cc:347] Connected to a master server at 127.11.8.254:38593
I20260926 16:33:09.926945 21888 heartbeater.cc:464] Registering TS with master...
I20260926 16:33:09.927486 21888 heartbeater.cc:511] Master 127.11.8.254:38593 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.195@KRBTEST.COM" created.
I20260926 16:33:09.928566 21563 ts_manager.cc:194] Registered new tserver with Master: b746c882da704aba801a1b1eb5982c97 (127.11.8.194:45757)
I20260926 16:33:09.929140 21563 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.194@KRBTEST.COM'} at 127.11.8.194:56367
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab.
Entry for principal kudu/127.11.8.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab.
I20260926 16:33:09.943274 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.195:0
--local_ip_for_outbound_sockets=127.11.8.195
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:38593
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:37235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:33:10.052546 21893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:33:10.052776 21893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:33:10.052816 21893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:33:10.056773 21893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:33:10.056824 21893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:33:10.056927 21893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.195
I20260926 16:33:10.062058 21893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:37235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:38593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.195
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:33:10.063100 21893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:33:10.064111 21893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:33:10.073771 21901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:33:10.073802 21899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:33:10.073952 21893 server_base.cc:1034] running on GCE node
W20260926 16:33:10.073792 21898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:33:10.074385 21893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:33:10.074934 21893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:33:10.076097 21893 hybrid_clock.cc:648] HybridClock initialized: now 1790440390076083 us; error 35 us; skew 500 ppm
Sep 26 16:33:10 dist-test-slave-2zvq krb5kdc[21518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440390, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:33:10.078997 21893 init.cc:377] Logged in from keytab as kudu/127.11.8.195@KRBTEST.COM (short username kudu)
I20260926 16:33:10.079782 21893 webserver.cc:460] Webserver started at http://127.11.8.195:46865/ using document root <none> and password file <none>
I20260926 16:33:10.080271 21893 fs_manager.cc:359] Metadata directory not provided
I20260926 16:33:10.080325 21893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:33:10.080476 21893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:33:10.082404 21893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/data/instance:
uuid: "a51ab29e88c0443eb6dc61db611d7cee"
format_stamp: "Formatted at 2026-09-26 16:33:10 on dist-test-slave-2zvq"
server_key: "e7c87a4bdda038063eea31b1833f9ad1"
server_key_iv: "50bec1d91733017317c3ef61de2c4b65"
server_key_version: "encryptionkey@0"
I20260926 16:33:10.082913 21893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance:
uuid: "a51ab29e88c0443eb6dc61db611d7cee"
format_stamp: "Formatted at 2026-09-26 16:33:10 on dist-test-slave-2zvq"
server_key: "e7c87a4bdda038063eea31b1833f9ad1"
server_key_iv: "50bec1d91733017317c3ef61de2c4b65"
server_key_version: "encryptionkey@0"
I20260926 16:33:10.086349 21893 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260926 16:33:10.088516 21908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:33:10.089673 21893 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260926 16:33:10.089764 21893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/wal
uuid: "a51ab29e88c0443eb6dc61db611d7cee"
format_stamp: "Formatted at 2026-09-26 16:33:10 on dist-test-slave-2zvq"
server_key: "e7c87a4bdda038063eea31b1833f9ad1"
server_key_iv: "50bec1d91733017317c3ef61de2c4b65"
server_key_version: "encryptionkey@0"
I20260926 16:33:10.089840 21893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:33:10.105788 21893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:33:10.108570 21893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:33:10.108767 21893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:33:10.109325 21893 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:33:10.110098 21893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:33:10.110168 21893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:33:10.110230 21893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:33:10.110319 21893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:33:10.118654 21893 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.195:41577
I20260926 16:33:10.118724 22021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.195:41577 every 8 connection(s)
I20260926 16:33:10.119529 21893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
Sep 26 16:33:10 dist-test-slave-2zvq krb5kdc[21518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440390, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:33:10.129379 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 21893
I20260926 16:33:10.129503 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance
I20260926 16:33:10.129830 11299 external_mini_cluster.cc:1428] Setting key cde25061f78a122c14c01b9ba915b0fb
I20260926 16:33:10.133121 21602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:10.121319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:34673 (local address 127.11.8.254:38593)
0926 16:33:10.121460 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:10.121463 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:10.122205 (+ 742us) server_negotiation.cc:407] Connection header received
0926 16:33:10.123642 (+ 1437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:10.123645 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:10.123704 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:10.123810 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:33:10.124933 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:10.125465 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:10.126677 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:10.126900 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:10.127008 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:10.129376 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:33:10.129380 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:33:10.129382 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:33:10.129411 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:33:10.131217 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:33:10.131775 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:33:10.131777 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:33:10.131779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:33:10.131850 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:33:10.132244 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:33:10.132247 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:33:10.132248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:33:10.132411 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:33:10.132487 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 16:33:10.132914 (+ 427us) server_negotiation.cc:299] Negotiation successful
0926 16:33:10.133006 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 16:33:10.133893 22024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:10.121613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:38593 (local address 127.11.8.195:34673)
0926 16:33:10.122053 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:10.122093 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 16:33:10.123430 (+ 1337us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:33:10.123982 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:33:10.123989 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:33:10.124331 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:33:10.124782 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:10.124794 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:10.125601 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:10.125605 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:10.126553 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:10.126562 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:10.127051 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:10.127055 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:10.127215 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:10.127803 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:33:10.127819 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:33:10.129221 (+ 1402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:33:10.131318 (+ 2097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:33:10.131324 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:33:10.131337 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:33:10.131683 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:33:10.131970 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:33:10.131973 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:33:10.131974 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:33:10.132158 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:33:10.132497 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:33:10.132515 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:33:10.132721 (+ 206us) client_negotiation.cc:769] Sending connection context
0926 16:33:10.132870 (+ 149us) client_negotiation.cc:240] Negotiation successful
0926 16:33:10.133096 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":109,"threads_started":1}
I20260926 16:33:10.134958 22022 heartbeater.cc:347] Connected to a master server at 127.11.8.254:38593
I20260926 16:33:10.135207 22022 heartbeater.cc:464] Registering TS with master...
I20260926 16:33:10.135667 22022 heartbeater.cc:511] Master 127.11.8.254:38593 requested a full tablet report, sending...
I20260926 16:33:10.136606 21563 ts_manager.cc:194] Registered new tserver with Master: a51ab29e88c0443eb6dc61db611d7cee (127.11.8.195:41577)
I20260926 16:33:10.137169 21563 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.195@KRBTEST.COM'} at 127.11.8.195:34673
I20260926 16:33:10.143615 11299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 16:33:10.152245 21602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:10.145450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44984 (local address 127.11.8.254:38593)
0926 16:33:10.145598 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:10.145604 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:10.145693 (+ 89us) server_negotiation.cc:407] Connection header received
0926 16:33:10.145783 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:10.145785 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:10.145819 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:10.145881 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:33:10.146527 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:10.147007 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:10.147844 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:10.148042 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:10.148114 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:10.149437 (+ 1323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:33:10.149440 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:33:10.149442 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:33:10.149460 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:33:10.150692 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:33:10.151138 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:33:10.151140 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:33:10.151141 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:33:10.151209 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:33:10.151439 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:33:10.151441 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:33:10.151443 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:33:10.151643 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:33:10.151745 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 16:33:10.151976 (+ 231us) server_negotiation.cc:299] Negotiation successful
0926 16:33:10.152063 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 16:33:10.154994 21563 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44984:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:33:10.158046 21563 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:33:10.171711 22033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:10.165779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:41577 (local address 127.0.0.1:35804)
0926 16:33:10.166115 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:10.166165 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 16:33:10.166312 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:33:10.167193 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:33:10.167201 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:33:10.167231 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:33:10.167375 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:10.167385 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:10.168804 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:10.168811 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:10.170074 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:10.170084 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:10.171157 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:10.171161 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:10.171269 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:10.171304 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 16:33:10.171436 (+ 132us) client_negotiation.cc:240] Negotiation successful
0926 16:33:10.171554 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":27,"thread_start_us":70,"threads_started":1}
I20260926 16:33:10.171888 22036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:10.166569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46372 (local address 127.11.8.193:40121)
0926 16:33:10.167234 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:10.167248 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:10.167442 (+ 194us) server_negotiation.cc:407] Connection header received
0926 16:33:10.167541 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:10.167547 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:10.167684 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:10.167870 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:33:10.168026 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:10.169086 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:10.170562 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:10.171322 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:10.171432 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:10.171578 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0926 16:33:10.171639 (+ 61us) server_negotiation.cc:299] Negotiation successful
0926 16:33:10.171727 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":132,"threads_started":1}
I20260926 16:33:10.172163 22034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:10.165914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35804 (local address 127.11.8.195:41577)
0926 16:33:10.166509 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:10.166525 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:10.166550 (+ 25us) server_negotiation.cc:407] Connection header received
0926 16:33:10.166629 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:10.166636 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:10.167011 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:10.167548 (+ 537us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:33:10.167688 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:10.168685 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:10.170208 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:10.171041 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:10.171155 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:10.171296 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0926 16:33:10.171905 (+ 609us) server_negotiation.cc:299] Negotiation successful
0926 16:33:10.172011 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":160,"threads_started":1}
I20260926 16:33:10.172286 22035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:10.165951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:40121 (local address 127.0.0.1:46372)
0926 16:33:10.167236 (+ 1285us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:10.167251 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:33:10.167345 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:33:10.167809 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:33:10.167812 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:33:10.167830 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:33:10.167933 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:10.167939 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:10.169235 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:10.169238 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:10.170392 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:10.170399 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:10.171443 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:10.171447 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:10.171535 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:10.171555 (+ 20us) client_negotiation.cc:769] Sending connection context
0926 16:33:10.171615 (+ 60us) client_negotiation.cc:240] Negotiation successful
0926 16:33:10.171668 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":840,"thread_start_us":118,"threads_started":1}
I20260926 16:33:10.173277 22037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:10.166652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:45757 (local address 127.0.0.1:56740)
0926 16:33:10.167365 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:10.167379 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:33:10.167449 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:33:10.168201 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:33:10.168204 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:33:10.168219 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:33:10.168293 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:10.168298 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:10.169612 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:10.169616 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:10.170794 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:10.170807 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:10.172023 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:10.172027 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:10.172141 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:10.172172 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 16:33:10.173081 (+ 909us) client_negotiation.cc:240] Negotiation successful
0926 16:33:10.173160 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"thread_start_us":92,"threads_started":1}
I20260926 16:33:10.174600 22038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:10.167098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56740 (local address 127.11.8.194:45757)
0926 16:33:10.167548 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:10.167560 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:10.167580 (+ 20us) server_negotiation.cc:407] Connection header received
0926 16:33:10.167650 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:10.167656 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:10.167838 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:10.168055 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:33:10.168419 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:10.169468 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:10.170927 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:10.171710 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:10.171815 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:10.171937 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0926 16:33:10.174352 (+ 2415us) server_negotiation.cc:299] Negotiation successful
0926 16:33:10.174454 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":127,"threads_started":1}
I20260926 16:33:10.175024 21956 tablet_service.cc:1467] Processing CreateTablet for tablet d8eed07935d84f09a37f2586e3984ed3 (DEFAULT_TABLE table=test-table [id=e570cf1ae012472c92eb7d39b05b918d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:33:10.175034 21688 tablet_service.cc:1467] Processing CreateTablet for tablet d8eed07935d84f09a37f2586e3984ed3 (DEFAULT_TABLE table=test-table [id=e570cf1ae012472c92eb7d39b05b918d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:33:10.176039 21956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8eed07935d84f09a37f2586e3984ed3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:33:10.176046 21688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8eed07935d84f09a37f2586e3984ed3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:33:10.177086 21822 tablet_service.cc:1467] Processing CreateTablet for tablet d8eed07935d84f09a37f2586e3984ed3 (DEFAULT_TABLE table=test-table [id=e570cf1ae012472c92eb7d39b05b918d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:33:10.177951 21822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8eed07935d84f09a37f2586e3984ed3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:33:10.182348 22039 tablet_bootstrap.cc:492] T d8eed07935d84f09a37f2586e3984ed3 P a51ab29e88c0443eb6dc61db611d7cee: Bootstrap starting.
I20260926 16:33:10.183655 22040 tablet_bootstrap.cc:492] T d8eed07935d84f09a37f2586e3984ed3 P ebc9a5c407fe44e3bd281fb34bb3d251: Bootstrap starting.
I20260926 16:33:10.184401 22041 tablet_bootstrap.cc:492] T d8eed07935d84f09a37f2586e3984ed3 P b746c882da704aba801a1b1eb5982c97: Bootstrap starting.
I20260926 16:33:10.184502 22039 tablet_bootstrap.cc:654] T d8eed07935d84f09a37f2586e3984ed3 P a51ab29e88c0443eb6dc61db611d7cee: Neither blocks nor log segments found. Creating new log.
I20260926 16:33:10.185726 22040 tablet_bootstrap.cc:654] T d8eed07935d84f09a37f2586e3984ed3 P ebc9a5c407fe44e3bd281fb34bb3d251: Neither blocks nor log segments found. Creating new log.
I20260926 16:33:10.185855 22039 log.cc:824] T d8eed07935d84f09a37f2586e3984ed3 P a51ab29e88c0443eb6dc61db611d7cee: Log is configured to *not* fsync() on all Append() calls
I20260926 16:33:10.187006 22041 tablet_bootstrap.cc:654] T d8eed07935d84f09a37f2586e3984ed3 P b746c882da704aba801a1b1eb5982c97: Neither blocks nor log segments found. Creating new log.
I20260926 16:33:10.186998 22040 log.cc:824] T d8eed07935d84f09a37f2586e3984ed3 P ebc9a5c407fe44e3bd281fb34bb3d251: Log is configured to *not* fsync() on all Append() calls
I20260926 16:33:10.187822 22041 log.cc:824] T d8eed07935d84f09a37f2586e3984ed3 P b746c882da704aba801a1b1eb5982c97: Log is configured to *not* fsync() on all Append() calls
I20260926 16:33:10.188836 22039 tablet_bootstrap.cc:492] T d8eed07935d84f09a37f2586e3984ed3 P a51ab29e88c0443eb6dc61db611d7cee: No bootstrap required, opened a new log
I20260926 16:33:10.188992 22040 tablet_bootstrap.cc:492] T d8eed07935d84f09a37f2586e3984ed3 P ebc9a5c407fe44e3bd281fb34bb3d251: No bootstrap required, opened a new log
I20260926 16:33:10.189034 22039 ts_tablet_manager.cc:1397] T d8eed07935d84f09a37f2586e3984ed3 P a51ab29e88c0443eb6dc61db611d7cee: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260926 16:33:10.189203 22040 ts_tablet_manager.cc:1397] T d8eed07935d84f09a37f2586e3984ed3 P ebc9a5c407fe44e3bd281fb34bb3d251: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 16:33:10.191828 22041 tablet_bootstrap.cc:492] T d8eed07935d84f09a37f2586e3984ed3 P b746c882da704aba801a1b1eb5982c97: No bootstrap required, opened a new log
I20260926 16:33:10.192010 22041 ts_tablet_manager.cc:1397] T d8eed07935d84f09a37f2586e3984ed3 P b746c882da704aba801a1b1eb5982c97: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260926 16:33:10.192188 22040 raft_consensus.cc:348] T d8eed07935d84f09a37f2586e3984ed3 P ebc9a5c407fe44e3bd281fb34bb3d251 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a51ab29e88c0443eb6dc61db611d7cee" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 41577 } } peers { permanent_uuid: "ebc9a5c407fe44e3bd281fb34bb3d251" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40121 } } peers { permanent_uuid: "b746c882da704aba801a1b1eb5982c97" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45757 } }
I20260926 16:33:10.192440 22040 raft_consensus.cc:374] T d8eed07935d84f09a37f2586e3984ed3 P ebc9a5c407fe44e3bd281fb34bb3d251 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:33:10.192513 22040 raft_consensus.cc:729] T d8eed07935d84f09a37f2586e3984ed3 P ebc9a5c407fe44e3bd281fb34bb3d251 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebc9a5c407fe44e3bd281fb34bb3d251, State: Initialized, Role: FOLLOWER
I20260926 16:33:10.192552 22039 raft_consensus.cc:348] T d8eed07935d84f09a37f2586e3984ed3 P a51ab29e88c0443eb6dc61db611d7cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a51ab29e88c0443eb6dc61db611d7cee" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 41577 } } peers { permanent_uuid: "ebc9a5c407fe44e3bd281fb34bb3d251" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40121 } } peers { permanent_uuid: "b746c882da704aba801a1b1eb5982c97" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45757 } }
I20260926 16:33:10.192806 22039 raft_consensus.cc:374] T d8eed07935d84f09a37f2586e3984ed3 P a51ab29e88c0443eb6dc61db611d7cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:33:10.192863 22039 raft_consensus.cc:729] T d8eed07935d84f09a37f2586e3984ed3 P a51ab29e88c0443eb6dc61db611d7cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a51ab29e88c0443eb6dc61db611d7cee, State: Initialized, Role: FOLLOWER
I20260926 16:33:10.193011 22040 consensus_queue.cc:260] T d8eed07935d84f09a37f2586e3984ed3 P ebc9a5c407fe44e3bd281fb34bb3d251 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a51ab29e88c0443eb6dc61db611d7cee" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 41577 } } peers { permanent_uuid: "ebc9a5c407fe44e3bd281fb34bb3d251" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40121 } } peers { permanent_uuid: "b746c882da704aba801a1b1eb5982c97" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45757 } }
I20260926 16:33:10.193292 22039 consensus_queue.cc:260] T d8eed07935d84f09a37f2586e3984ed3 P a51ab29e88c0443eb6dc61db611d7cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a51ab29e88c0443eb6dc61db611d7cee" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 41577 } } peers { permanent_uuid: "ebc9a5c407fe44e3bd281fb34bb3d251" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40121 } } peers { permanent_uuid: "b746c882da704aba801a1b1eb5982c97" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45757 } }
I20260926 16:33:10.193645 21754 heartbeater.cc:503] Master 127.11.8.254:38593 was elected leader, sending a full tablet report...
I20260926 16:33:10.194463 22039 ts_tablet_manager.cc:1428] T d8eed07935d84f09a37f2586e3984ed3 P a51ab29e88c0443eb6dc61db611d7cee: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260926 16:33:10.194836 22022 heartbeater.cc:503] Master 127.11.8.254:38593 was elected leader, sending a full tablet report...
I20260926 16:33:10.195091 22040 ts_tablet_manager.cc:1428] T d8eed07935d84f09a37f2586e3984ed3 P ebc9a5c407fe44e3bd281fb34bb3d251: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260926 16:33:10.196605 22041 raft_consensus.cc:348] T d8eed07935d84f09a37f2586e3984ed3 P b746c882da704aba801a1b1eb5982c97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a51ab29e88c0443eb6dc61db611d7cee" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 41577 } } peers { permanent_uuid: "ebc9a5c407fe44e3bd281fb34bb3d251" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40121 } } peers { permanent_uuid: "b746c882da704aba801a1b1eb5982c97" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45757 } }
I20260926 16:33:10.196857 22041 raft_consensus.cc:374] T d8eed07935d84f09a37f2586e3984ed3 P b746c882da704aba801a1b1eb5982c97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:33:10.196943 22041 raft_consensus.cc:729] T d8eed07935d84f09a37f2586e3984ed3 P b746c882da704aba801a1b1eb5982c97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b746c882da704aba801a1b1eb5982c97, State: Initialized, Role: FOLLOWER
I20260926 16:33:10.197381 22041 consensus_queue.cc:260] T d8eed07935d84f09a37f2586e3984ed3 P b746c882da704aba801a1b1eb5982c97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a51ab29e88c0443eb6dc61db611d7cee" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 41577 } } peers { permanent_uuid: "ebc9a5c407fe44e3bd281fb34bb3d251" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40121 } } peers { permanent_uuid: "b746c882da704aba801a1b1eb5982c97" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45757 } }
I20260926 16:33:10.198135 21888 heartbeater.cc:503] Master 127.11.8.254:38593 was elected leader, sending a full tablet report...
I20260926 16:33:10.198349 22041 ts_tablet_manager.cc:1428] T d8eed07935d84f09a37f2586e3984ed3 P b746c882da704aba801a1b1eb5982c97: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260926 16:33:10.201448 21755 tablet.cc:2369] T d8eed07935d84f09a37f2586e3984ed3 P ebc9a5c407fe44e3bd281fb34bb3d251: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:33:10.224212 22045 raft_consensus.cc:482] T d8eed07935d84f09a37f2586e3984ed3 P ebc9a5c407fe44e3bd281fb34bb3d251 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:33:10.224370 22045 raft_consensus.cc:504] T d8eed07935d84f09a37f2586e3984ed3 P ebc9a5c407fe44e3bd281fb34bb3d251 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a51ab29e88c0443eb6dc61db611d7cee" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 41577 } } peers { permanent_uuid: "ebc9a5c407fe44e3bd281fb34bb3d251" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40121 } } peers { permanent_uuid: "b746c882da704aba801a1b1eb5982c97" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45757 } }
I20260926 16:33:10.225566 22045 leader_election.cc:288] T d8eed07935d84f09a37f2586e3984ed3 P ebc9a5c407fe44e3bd281fb34bb3d251 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a51ab29e88c0443eb6dc61db611d7cee (127.11.8.195:41577), b746c882da704aba801a1b1eb5982c97 (127.11.8.194:45757)
I20260926 16:33:10.229409 22038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:10.225874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:46437 (local address 127.11.8.194:45757)
0926 16:33:10.226158 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:10.226163 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:10.226179 (+ 16us) server_negotiation.cc:407] Connection header received
0926 16:33:10.226298 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:10.226301 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:10.226356 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:10.226432 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:33:10.226734 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:10.227276 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:10.228448 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:10.228836 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:10.228958 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:10.229012 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 16:33:10.229214 (+ 202us) server_negotiation.cc:299] Negotiation successful
0926 16:33:10.229286 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 16:33:10.229420 22034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:10.225909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:55213 (local address 127.11.8.195:41577)
0926 16:33:10.226057 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:10.226064 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:10.226298 (+ 234us) server_negotiation.cc:407] Connection header received
0926 16:33:10.226416 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:10.226422 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:10.226475 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:10.226553 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:33:10.226822 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:10.227291 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:10.228439 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:10.228844 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:10.228974 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:10.229022 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 16:33:10.229252 (+ 230us) server_negotiation.cc:299] Negotiation successful
0926 16:33:10.229320 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 16:33:10.229436 22049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:10.225776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:41577 (local address 127.11.8.193:55213)
0926 16:33:10.226171 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:10.226186 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:33:10.226311 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:33:10.226579 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:33:10.226581 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:33:10.226594 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:33:10.226711 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:10.226720 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:10.227426 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:10.227430 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:10.228311 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:10.228321 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:10.228976 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:10.228979 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:10.229107 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:10.229124 (+ 17us) client_negotiation.cc:769] Sending connection context
0926 16:33:10.229223 (+ 99us) client_negotiation.cc:240] Negotiation successful
0926 16:33:10.229286 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":63,"thread_start_us":112,"threads_started":1}
I20260926 16:33:10.229442 22048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:10.225776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:45757 (local address 127.11.8.193:46437)
0926 16:33:10.226087 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:10.226099 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:33:10.226174 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:33:10.226459 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:33:10.226463 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:33:10.226480 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:33:10.226611 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:10.226621 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:10.227394 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:10.227399 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:10.228314 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:10.228323 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:10.228959 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:10.228963 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:10.229068 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:10.229095 (+ 27us) client_negotiation.cc:769] Sending connection context
0926 16:33:10.229166 (+ 71us) client_negotiation.cc:240] Negotiation successful
0926 16:33:10.229252 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":81,"thread_start_us":73,"threads_started":1}
I20260926 16:33:10.230005 21976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8eed07935d84f09a37f2586e3984ed3" candidate_uuid: "ebc9a5c407fe44e3bd281fb34bb3d251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a51ab29e88c0443eb6dc61db611d7cee" is_pre_election: true
I20260926 16:33:10.230304 21976 raft_consensus.cc:2454] T d8eed07935d84f09a37f2586e3984ed3 P a51ab29e88c0443eb6dc61db611d7cee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ebc9a5c407fe44e3bd281fb34bb3d251 in term 0.
I20260926 16:33:10.230396 21842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8eed07935d84f09a37f2586e3984ed3" candidate_uuid: "ebc9a5c407fe44e3bd281fb34bb3d251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b746c882da704aba801a1b1eb5982c97" is_pre_election: true
I20260926 16:33:10.230702 21842 raft_consensus.cc:2454] T d8eed07935d84f09a37f2586e3984ed3 P b746c882da704aba801a1b1eb5982c97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ebc9a5c407fe44e3bd281fb34bb3d251 in term 0.
I20260926 16:33:10.230763 21644 leader_election.cc:302] T d8eed07935d84f09a37f2586e3984ed3 P ebc9a5c407fe44e3bd281fb34bb3d251 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a51ab29e88c0443eb6dc61db611d7cee, ebc9a5c407fe44e3bd281fb34bb3d251; no voters:
I20260926 16:33:10.231007 22045 raft_consensus.cc:2781] T d8eed07935d84f09a37f2586e3984ed3 P ebc9a5c407fe44e3bd281fb34bb3d251 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:33:10.231098 22045 raft_consensus.cc:482] T d8eed07935d84f09a37f2586e3984ed3 P ebc9a5c407fe44e3bd281fb34bb3d251 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:33:10.231163 22045 raft_consensus.cc:3037] T d8eed07935d84f09a37f2586e3984ed3 P ebc9a5c407fe44e3bd281fb34bb3d251 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:33:10.232131 22045 raft_consensus.cc:504] T d8eed07935d84f09a37f2586e3984ed3 P ebc9a5c407fe44e3bd281fb34bb3d251 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a51ab29e88c0443eb6dc61db611d7cee" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 41577 } } peers { permanent_uuid: "ebc9a5c407fe44e3bd281fb34bb3d251" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40121 } } peers { permanent_uuid: "b746c882da704aba801a1b1eb5982c97" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45757 } }
I20260926 16:33:10.232484 22045 leader_election.cc:288] T d8eed07935d84f09a37f2586e3984ed3 P ebc9a5c407fe44e3bd281fb34bb3d251 [CANDIDATE]: Term 1 election: Requested vote from peers a51ab29e88c0443eb6dc61db611d7cee (127.11.8.195:41577), b746c882da704aba801a1b1eb5982c97 (127.11.8.194:45757)
I20260926 16:33:10.232733 21976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8eed07935d84f09a37f2586e3984ed3" candidate_uuid: "ebc9a5c407fe44e3bd281fb34bb3d251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a51ab29e88c0443eb6dc61db611d7cee"
I20260926 16:33:10.232827 21842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8eed07935d84f09a37f2586e3984ed3" candidate_uuid: "ebc9a5c407fe44e3bd281fb34bb3d251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b746c882da704aba801a1b1eb5982c97"
I20260926 16:33:10.232863 21976 raft_consensus.cc:3037] T d8eed07935d84f09a37f2586e3984ed3 P a51ab29e88c0443eb6dc61db611d7cee [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:33:10.232954 21842 raft_consensus.cc:3037] T d8eed07935d84f09a37f2586e3984ed3 P b746c882da704aba801a1b1eb5982c97 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:33:10.233862 21976 raft_consensus.cc:2454] T d8eed07935d84f09a37f2586e3984ed3 P a51ab29e88c0443eb6dc61db611d7cee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebc9a5c407fe44e3bd281fb34bb3d251 in term 1.
I20260926 16:33:10.233927 21842 raft_consensus.cc:2454] T d8eed07935d84f09a37f2586e3984ed3 P b746c882da704aba801a1b1eb5982c97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebc9a5c407fe44e3bd281fb34bb3d251 in term 1.
I20260926 16:33:10.234223 21642 leader_election.cc:302] T d8eed07935d84f09a37f2586e3984ed3 P ebc9a5c407fe44e3bd281fb34bb3d251 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b746c882da704aba801a1b1eb5982c97, ebc9a5c407fe44e3bd281fb34bb3d251; no voters:
I20260926 16:33:10.234416 22045 raft_consensus.cc:2781] T d8eed07935d84f09a37f2586e3984ed3 P ebc9a5c407fe44e3bd281fb34bb3d251 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:33:10.234620 22045 raft_consensus.cc:686] T d8eed07935d84f09a37f2586e3984ed3 P ebc9a5c407fe44e3bd281fb34bb3d251 [term 1 LEADER]: Becoming Leader. State: Replica: ebc9a5c407fe44e3bd281fb34bb3d251, State: Running, Role: LEADER
I20260926 16:33:10.234918 22045 consensus_queue.cc:237] T d8eed07935d84f09a37f2586e3984ed3 P ebc9a5c407fe44e3bd281fb34bb3d251 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a51ab29e88c0443eb6dc61db611d7cee" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 41577 } } peers { permanent_uuid: "ebc9a5c407fe44e3bd281fb34bb3d251" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40121 } } peers { permanent_uuid: "b746c882da704aba801a1b1eb5982c97" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45757 } }
I20260926 16:33:10.237895 21563 catalog_manager.cc:5495] T d8eed07935d84f09a37f2586e3984ed3 P ebc9a5c407fe44e3bd281fb34bb3d251 reported cstate change: term changed from 0 to 1, leader changed from <none> to ebc9a5c407fe44e3bd281fb34bb3d251 (127.11.8.193). New cstate: current_term: 1 leader_uuid: "ebc9a5c407fe44e3bd281fb34bb3d251" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a51ab29e88c0443eb6dc61db611d7cee" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 41577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ebc9a5c407fe44e3bd281fb34bb3d251" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 40121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b746c882da704aba801a1b1eb5982c97" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45757 } health_report { overall_health: UNKNOWN } } }
I20260926 16:33:10.247792 22036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:10.244558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46378 (local address 127.11.8.193:40121)
0926 16:33:10.244665 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:10.244669 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:10.244680 (+ 11us) server_negotiation.cc:407] Connection header received
0926 16:33:10.244770 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:10.244772 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:10.244816 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:10.244899 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:33:10.245198 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:10.245667 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:10.246506 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:10.246705 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:10.246781 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:10.247108 (+ 327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:33:10.247471 (+ 363us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:33:10.247558 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 16:33:10.247655 (+ 97us) server_negotiation.cc:299] Negotiation successful
0926 16:33:10.247696 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 16:33:10.253055 21976 raft_consensus.cc:1261] T d8eed07935d84f09a37f2586e3984ed3 P a51ab29e88c0443eb6dc61db611d7cee [term 1 FOLLOWER]: Refusing update from remote peer ebc9a5c407fe44e3bd281fb34bb3d251: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:33:10.253075 21842 raft_consensus.cc:1261] T d8eed07935d84f09a37f2586e3984ed3 P b746c882da704aba801a1b1eb5982c97 [term 1 FOLLOWER]: Refusing update from remote peer ebc9a5c407fe44e3bd281fb34bb3d251: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:33:10.253731 22045 consensus_queue.cc:1035] T d8eed07935d84f09a37f2586e3984ed3 P ebc9a5c407fe44e3bd281fb34bb3d251 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b746c882da704aba801a1b1eb5982c97" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 45757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:33:10.253935 22050 consensus_queue.cc:1035] T d8eed07935d84f09a37f2586e3984ed3 P ebc9a5c407fe44e3bd281fb34bb3d251 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a51ab29e88c0443eb6dc61db611d7cee" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 41577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:33:10.261550 22055 mvcc.cc:205] Tried to move back new op lower bound from 7333643838470430720 to 7333643838404685824. Current Snapshot: MvccSnapshot[applied={T|T < 7333643838470430720}]
I20260926 16:33:10.262089 22053 mvcc.cc:205] Tried to move back new op lower bound from 7333643838470430720 to 7333643838404685824. Current Snapshot: MvccSnapshot[applied={T|T < 7333643838470430720}]
I20260926 16:33:10.269678 21563 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:33:10.269840 21563 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:33:10.272133 21563 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 34789ec4754d444caecc6628ad67452d: Sending DeleteTablet for 3 replicas of tablet d8eed07935d84f09a37f2586e3984ed3
I20260926 16:33:10.272821 21956 tablet_service.cc:1514] Processing DeleteTablet for tablet d8eed07935d84f09a37f2586e3984ed3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:33:10 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:35804
I20260926 16:33:10.272931 21688 tablet_service.cc:1514] Processing DeleteTablet for tablet d8eed07935d84f09a37f2586e3984ed3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:33:10 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:46372
I20260926 16:33:10.273393 21822 tablet_service.cc:1514] Processing DeleteTablet for tablet d8eed07935d84f09a37f2586e3984ed3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:33:10 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:56740
I20260926 16:33:10.273633 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 21625
I20260926 16:33:10.274566 22065 tablet_replica.cc:330] T d8eed07935d84f09a37f2586e3984ed3 P b746c882da704aba801a1b1eb5982c97: stopping tablet replica
I20260926 16:33:10.274848 22065 raft_consensus.cc:2229] T d8eed07935d84f09a37f2586e3984ed3 P b746c882da704aba801a1b1eb5982c97 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:33:10.273546 22063 tablet_replica.cc:330] T d8eed07935d84f09a37f2586e3984ed3 P a51ab29e88c0443eb6dc61db611d7cee: stopping tablet replica
I20260926 16:33:10.275180 22065 raft_consensus.cc:2258] T d8eed07935d84f09a37f2586e3984ed3 P b746c882da704aba801a1b1eb5982c97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:33:10.275318 22063 raft_consensus.cc:2229] T d8eed07935d84f09a37f2586e3984ed3 P a51ab29e88c0443eb6dc61db611d7cee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:33:10.275645 22063 raft_consensus.cc:2258] T d8eed07935d84f09a37f2586e3984ed3 P a51ab29e88c0443eb6dc61db611d7cee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:33:10.276976 22065 ts_tablet_manager.cc:1905] T d8eed07935d84f09a37f2586e3984ed3 P b746c882da704aba801a1b1eb5982c97: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:33:10.277240 22063 ts_tablet_manager.cc:1905] T d8eed07935d84f09a37f2586e3984ed3 P a51ab29e88c0443eb6dc61db611d7cee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:33:10.279392 22063 ts_tablet_manager.cc:1918] T d8eed07935d84f09a37f2586e3984ed3 P a51ab29e88c0443eb6dc61db611d7cee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:33:10.279507 22063 log.cc:1196] T d8eed07935d84f09a37f2586e3984ed3 P a51ab29e88c0443eb6dc61db611d7cee: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-2/wal/wals/d8eed07935d84f09a37f2586e3984ed3
I20260926 16:33:10.279803 22063 ts_tablet_manager.cc:1939] T d8eed07935d84f09a37f2586e3984ed3 P a51ab29e88c0443eb6dc61db611d7cee: Deleting consensus metadata
I20260926 16:33:10.280364 22065 ts_tablet_manager.cc:1918] T d8eed07935d84f09a37f2586e3984ed3 P b746c882da704aba801a1b1eb5982c97: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:33:10.280467 22065 log.cc:1196] T d8eed07935d84f09a37f2586e3984ed3 P b746c882da704aba801a1b1eb5982c97: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790440324823152-11299-0/minicluster-data/ts-1/wal/wals/d8eed07935d84f09a37f2586e3984ed3
I20260926 16:33:10.280857 22065 ts_tablet_manager.cc:1939] T d8eed07935d84f09a37f2586e3984ed3 P b746c882da704aba801a1b1eb5982c97: Deleting consensus metadata
I20260926 16:33:10.280962 21550 catalog_manager.cc:4841] TS a51ab29e88c0443eb6dc61db611d7cee (127.11.8.195:41577): tablet d8eed07935d84f09a37f2586e3984ed3 (table test-table [id=e570cf1ae012472c92eb7d39b05b918d]) successfully deleted
W20260926 16:33:10.281605 21548 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.8.193:40121 (error 108)
I20260926 16:33:10.282132 21548 catalog_manager.cc:4841] TS b746c882da704aba801a1b1eb5982c97 (127.11.8.194:45757): tablet d8eed07935d84f09a37f2586e3984ed3 (table test-table [id=e570cf1ae012472c92eb7d39b05b918d]) successfully deleted
W20260926 16:33:10.282675 22037 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:33:10.282442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:40121 (local address 127.0.0.1:46380)
0926 16:33:10.282545 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:10.282604 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.193:40121: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":26}
I20260926 16:33:10.282850 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 21759
W20260926 16:33:10.282832 21548 catalog_manager.cc:4584] TS ebc9a5c407fe44e3bd281fb34bb3d251 (127.11.8.193:40121): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d8eed07935d84f09a37f2586e3984ed3: Network error: Client connection negotiation failed: client connection to 127.11.8.193:40121: connect: Connection reset by peer (error 104)
I20260926 16:33:10.288216 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 21893
I20260926 16:33:10.293483 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 21531
2026-09-26T16:33:10Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2206 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:33:10 dist-test-slave-2zvq krb5kdc[22069](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:33:10 dist-test-slave-2zvq krb5kdc[22069](info): set up 2 sockets
Sep 26 16:33:10 dist-test-slave-2zvq krb5kdc[22069](info): commencing operation
krb5kdc: starting...
W20260926 16:33:11.328722 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:33:11 dist-test-slave-2zvq krb5kdc[22069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:33:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:33:11Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.11.8.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.11.8.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.11.8.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/krb5kdc/rangeradmin_127.11.8.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.11.8.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/krb5kdc/rangeradmin_127.11.8.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.11.8.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.11.8.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.11.8.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/krb5kdc/rangerlookup_127.11.8.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.11.8.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/krb5kdc/rangerlookup_127.11.8.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.8.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.11.8.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.8.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/krb5kdc/HTTP_127.11.8.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.8.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/krb5kdc/HTTP_127.11.8.212@KRBTEST.COM.keytab.
I20260926 16:33:11.492746 11299 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskTzWGcB/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/postgres -l logfile start
2026-09-26 16:33:13.994 UTC [22099] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-26 16:33:13.994 UTC [22099] LOG: listening on IPv4 address "127.11.8.212", port 46527
2026-09-26 16:33:13.998 UTC [22099] LOG: listening on Unix socket "/tmp/.s.PGSQL.46527"
2026-09-26 16:33:14.035 UTC [22102] LOG: database system was shut down at 2026-09-26 16:33:12 UTC
2026-09-26 16:33:14.039 UTC [22099] LOG: database system is ready to accept connections
I20260926 16:33:14.980818 11299 mini_postgres.cc:96] Postgres bound to 46527
2026-09-26 16:33:14.987 UTC [22110] FATAL: database "slave" does not exist
127.11.8.212:46527 - accepting connections
I20260926 16:33:14.987839 11299 mini_ranger.cc:161] Starting Ranger...
I20260926 16:33:15.002599 11299 mini_ranger.cc:84] Created miniranger Postgres user
I20260926 16:33:15.512830 11299 mini_ranger.cc:87] Created ranger Postgres database
I20260926 16:33:15.512928 11299 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ranger-admin
2026-09-26 16:33:15,812 [I] DB FLAVOR :POSTGRES
2026-09-26 16:33:15,813 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 16:33:15,813 [I] Checking connection..
2026-09-26 16:33:15,813 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:46527/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-26 16:33:16,066 [I] Checking connection passed.
2026-09-26 16:33:16,066 [I] --------- Verifying version history table ---------
2026-09-26 16:33:16,066 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:46527/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 16:33:16,293 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 16:33:16,294 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:46527/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 16:33:16,535 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 16:33:16,535 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-26 16:33:16,535 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:46527/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-26 16:33:16.761 UTC [22174] WARNING: there is no transaction in progress
2026-09-26 16:33:16,767 [I] create_dbversion_catalog.sql file imported successfully
2026-09-26 16:33:16,768 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:46527/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 16:33:16,997 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-26 16:33:16,997 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 16:33:16,998 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:46527/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-26 16:33:17,235 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:46527/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-26 16:33:17,466 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:46527/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-2zvq.c.gcp-upstream.internal','N') ;"
2026-09-26 16:33:17,698 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:46527/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 16:33:17,953 [I] Table x_portal_user does not exist in database ranger
2026-09-26 16:33:17,953 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:46527/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 16:33:18,184 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-26 16:33:18,184 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:46527/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 16:33:18,434 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:46527/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 16:33:18,681 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-26 16:33:18,681 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:46527/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-26 16:33:19.680 UTC [22282] WARNING: there is no transaction in progress
2026-09-26 16:33:19.696 UTC [22282] WARNING: there is no transaction in progress
2026-09-26 16:33:19.713 UTC [22282] WARNING: there is no transaction in progress
2026-09-26 16:33:19.728 UTC [22282] WARNING: there is no transaction in progress
2026-09-26 16:33:19.747 UTC [22282] WARNING: there is no transaction in progress
2026-09-26 16:33:19.764 UTC [22282] WARNING: there is no transaction in progress
2026-09-26 16:33:19.821 UTC [22282] WARNING: there is no transaction in progress
2026-09-26 16:33:19.828 UTC [22282] WARNING: there is no transaction in progress
2026-09-26 16:33:19.842 UTC [22282] WARNING: there is no transaction in progress
2026-09-26 16:33:19.853 UTC [22282] WARNING: there is no transaction in progress
2026-09-26 16:33:19.864 UTC [22282] WARNING: there is no transaction in progress
2026-09-26 16:33:19.878 UTC [22282] WARNING: there is no transaction in progress
2026-09-26 16:33:19.888 UTC [22282] WARNING: there is no transaction in progress
2026-09-26 16:33:19.895 UTC [22282] WARNING: there is no transaction in progress
2026-09-26 16:33:19.902 UTC [22282] WARNING: there is no transaction in progress
2026-09-26 16:33:19.910 UTC [22282] WARNING: there is no transaction in progress
2026-09-26 16:33:20,578 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-26 16:33:20,579 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:46527/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-26 16:33:20,816 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-26 16:33:20,817 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:46527/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 16:33:21,059 [I] Table x_portal_user already exists in database 'ranger'
2026-09-26 16:33:21,060 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:46527/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 16:33:21,309 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-26 16:33:21,310 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:46527/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 16:33:21,554 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:46527/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 16:33:21,784 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:46527/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-2zvq.c.gcp-upstream.internal';"
2026-09-26 16:33:22,030 [I] CORE_DB_SCHEMA import status has been updated
2026-09-26 16:33:22,030 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:46527/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-26 16:33:22,270 [I] DB_PATCHES have already been applied
I20260926 16:33:22.275635 11299 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ranger-admin:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskTzWGcB/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260926 16:33:22.275698 11299 mini_ranger.cc:193] Using host: 127.11.8.212
I20260926 16:33:22.279110 11299 mini_ranger.cc:227] Ranger admin URL: http://127.11.8.212:37855
Sep 26, 2026 4:33:22 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 4:33:22 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 4:33:22 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ranger-admin/ews/webapp
Sep 26, 2026 4:33:22 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 26, 2026 4:33:22 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 26, 2026 4:33:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ranger-admin/ews/webapp].
Sep 26, 2026 4:33:23 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260926 16:33:23.374754 11299 mini_ranger.cc:160] Time spent starting Ranger: real 8.387s user 0.001s sys 0.009s
Sep 26, 2026 4:33:43 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 26, 2026 4:33:43 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Sep 26, 2026 4:33:43 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 26, 2026 4:33:43 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 26, 2026 4:33:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 26, 2026 4:33:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 26, 2026 4:33:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 26, 2026 4:33:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 26, 2026 4:33:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 26, 2026 4:33:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 26, 2026 4:33:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 26, 2026 4:33:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 26, 2026 4:33:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 26, 2026 4:33:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 26, 2026 4:33:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 26, 2026 4:33:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 26, 2026 4:33:43 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 26, 2026 4:33:44 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260926 16:33:44.696084 11299 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.11.8.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.11.8.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.11.8.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/krb5kdc/rangerkms_127.11.8.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.11.8.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/krb5kdc/rangerkms_127.11.8.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.8.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.11.8.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.8.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/krb5kdc/HTTP_127.11.8.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.8.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/krb5kdc/HTTP_127.11.8.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 26 16:33:44 dist-test-slave-2zvq krb5kdc[22069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440424, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260926 16:33:44.779515 11299 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260926 16:33:44.789140 11299 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260926 16:33:45.646097 11299 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260926 16:33:45.646195 11299 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ranger-kms
I20260926 16:33:45.646332 11299 mini_ranger_kms.cc:227] Using postgres at 127.11.8.212:46527
2026-09-26 16:33:45,823 [I] DB FLAVOR :POSTGRES
2026-09-26 16:33:45,823 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 16:33:45,823 [I] Checking connection
2026-09-26 16:33:45,823 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:46527/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-26 16:33:46,077 [I] connection success
2026-09-26 16:33:46,077 [I] --------- Verifying Ranger DB tables ---------
2026-09-26 16:33:46,077 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-26 16:33:46,077 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:46527/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-26 16:33:46,328 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-26 16:33:46,328 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 16:33:46,329 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-26 16:33:46,329 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:46527/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-26 16:33:46,608 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260926 16:33:47.051587 11299 mini_ranger_kms.cc:310] Created kms service
I20260926 16:33:47.262110 11299 mini_ranger_kms.cc:326] Created kudu user
I20260926 16:33:47.305208 11299 mini_ranger_kms.cc:343] Created rangerkms user
Sep 26 16:33:47 dist-test-slave-2zvq krb5kdc[22069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440424, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.11.8.212@KRBTEST.COM
I20260926 16:33:47.703058 11299 mini_ranger_kms.cc:384] Added ranger policy
I20260926 16:33:47.703219 11299 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ranger-kms:/tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260926 16:33:47.703244 11299 mini_ranger_kms.cc:240] Using host: 127.11.8.212
I20260926 16:33:47.707089 11299 mini_ranger_kms.cc:276] Ranger KMS PID: 22462
I20260926 16:33:47.707227 11299 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.11.8.212:40681
Sep 26, 2026 4:33:48 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 4:33:48 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 4:33:48 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 26, 2026 4:33:48 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 26, 2026 4:33:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ranger-kms/ews/webapp].
Sep 26, 2026 4:33:49 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Sep 26 16:33:49 dist-test-slave-2zvq krb5kdc[22069](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790440429, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.8.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:33:49.991267 11299 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.212s user 0.023s sys 0.139s
I20260926 16:33:49.991398 11299 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260926 16:33:49.991518 11299 mini_ranger_kms.cc:401] 127.11.8.212:40681/kms/v1/keys
Sep 26 16:33:50 dist-test-slave-2zvq krb5kdc[22069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440429, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.8.212@KRBTEST.COM for HTTP/127.11.8.212@KRBTEST.COM
Sep 26 16:33:50 dist-test-slave-2zvq krb5kdc[22069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440429, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.8.212@KRBTEST.COM for HTTP/127.11.8.212@KRBTEST.COM
Sep 26 16:33:52 dist-test-slave-2zvq krb5kdc[22069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.11.8.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
I20260926 16:33:52.873435 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:40825
--webserver_interface=127.11.8.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:44825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:40825
--ranger_config_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.8.212:40681/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:33:52.997714 22513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:33:52.997987 22513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:33:52.998026 22513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:33:53.002027 22513 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:33:53.002086 22513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:33:53.002106 22513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:33:53.002122 22513 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:33:53.002182 22513 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:33:53.007644 22513 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:44825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/master-0/wal
--ranger_kms_url=127.11.8.212:40681/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:40825
--ranger_config_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:40825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22513
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:33:53.008677 22513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:33:53.009701 22513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:33:53.018617 22521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:33:53.018600 22519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:33:53.018654 22513 server_base.cc:1034] running on GCE node
W20260926 16:33:53.018599 22518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:33:53.019405 22513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:33:53.020412 22513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:33:53.021584 22513 hybrid_clock.cc:648] HybridClock initialized: now 1790440433021558 us; error 38 us; skew 500 ppm
Sep 26 16:33:53 dist-test-slave-2zvq krb5kdc[22069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440433, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:33:53.024305 22513 init.cc:377] Logged in from keytab as kudu/127.11.8.254@KRBTEST.COM (short username kudu)
I20260926 16:33:53.025168 22513 webserver.cc:460] Webserver started at http://127.11.8.254:45733/ using document root <none> and password file <none>
I20260926 16:33:53.025704 22513 fs_manager.cc:359] Metadata directory not provided
I20260926 16:33:53.025784 22513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:33:53.026046 22513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 16:33:53 dist-test-slave-2zvq krb5kdc[22069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440433, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for HTTP/127.11.8.212@KRBTEST.COM
I20260926 16:33:53.089329 22513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "48bfb21da5794ad3b3e0984679c66d07"
format_stamp: "Formatted at 2026-09-26 16:33:53 on dist-test-slave-2zvq"
server_key: "c74d4e509333bba56480e48c0ef72e34"
server_key_iv: "51b89759728c6ead8a55c27c2353b96e"
server_key_version: "kuduclusterkey@0"
I20260926 16:33:53.090123 22513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "48bfb21da5794ad3b3e0984679c66d07"
format_stamp: "Formatted at 2026-09-26 16:33:53 on dist-test-slave-2zvq"
server_key: "c74d4e509333bba56480e48c0ef72e34"
server_key_iv: "51b89759728c6ead8a55c27c2353b96e"
server_key_version: "kuduclusterkey@0"
I20260926 16:33:53.095222 22513 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260926 16:33:53.111992 22528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:33:53.113533 22513 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 16:33:53.113711 22513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "48bfb21da5794ad3b3e0984679c66d07"
format_stamp: "Formatted at 2026-09-26 16:33:53 on dist-test-slave-2zvq"
server_key: "c74d4e509333bba56480e48c0ef72e34"
server_key_iv: "51b89759728c6ead8a55c27c2353b96e"
server_key_version: "kuduclusterkey@0"
I20260926 16:33:53.113852 22513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:33:53.123577 22513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:33:53.126597 22513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:33:53.126794 22513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:33:53.134074 22513 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:40825
I20260926 16:33:53.134092 22580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:40825 every 8 connection(s)
I20260926 16:33:53.135113 22513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:33:53.138505 22581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:33:53.141320 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 22513
I20260926 16:33:53.141472 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:33:53.141759 11299 external_mini_cluster.cc:1428] Setting key ed67647ab919918f4eaacea624dd041e
I20260926 16:33:53.145547 22581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48bfb21da5794ad3b3e0984679c66d07: Bootstrap starting.
I20260926 16:33:53.148232 22581 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48bfb21da5794ad3b3e0984679c66d07: Neither blocks nor log segments found. Creating new log.
I20260926 16:33:53.149097 22581 log.cc:824] T 00000000000000000000000000000000 P 48bfb21da5794ad3b3e0984679c66d07: Log is configured to *not* fsync() on all Append() calls
Sep 26 16:33:53 dist-test-slave-2zvq krb5kdc[22069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:33:53.151309 22581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48bfb21da5794ad3b3e0984679c66d07: No bootstrap required, opened a new log
I20260926 16:33:53.155807 22584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:53.143271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40638 (local address 127.11.8.254:40825)
0926 16:33:53.143651 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:53.143671 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:53.143711 (+ 40us) server_negotiation.cc:407] Connection header received
0926 16:33:53.144359 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:53.144375 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:53.144706 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:53.145042 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:33:53.145869 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:53.146994 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:53.147864 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:53.148377 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:53.148569 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:53.151333 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:33:53.151339 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:33:53.151352 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:33:53.151401 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:33:53.153062 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:33:53.153506 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:33:53.153511 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:33:53.153517 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:33:53.153626 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:33:53.153911 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:33:53.153915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:33:53.153917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:33:53.154357 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:33:53.154566 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0926 16:33:53.154800 (+ 234us) server_negotiation.cc:299] Negotiation successful
0926 16:33:53.155093 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":128,"threads_started":1}
I20260926 16:33:53.155630 22581 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48bfb21da5794ad3b3e0984679c66d07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48bfb21da5794ad3b3e0984679c66d07" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 40825 } }
I20260926 16:33:53.155927 22581 raft_consensus.cc:374] T 00000000000000000000000000000000 P 48bfb21da5794ad3b3e0984679c66d07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:33:53.155975 22581 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48bfb21da5794ad3b3e0984679c66d07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48bfb21da5794ad3b3e0984679c66d07, State: Initialized, Role: FOLLOWER
I20260926 16:33:53.156452 22581 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48bfb21da5794ad3b3e0984679c66d07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48bfb21da5794ad3b3e0984679c66d07" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 40825 } }
I20260926 16:33:53.156610 22581 raft_consensus.cc:388] T 00000000000000000000000000000000 P 48bfb21da5794ad3b3e0984679c66d07 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:33:53.156693 22581 raft_consensus.cc:482] T 00000000000000000000000000000000 P 48bfb21da5794ad3b3e0984679c66d07 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:33:53.156785 22581 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48bfb21da5794ad3b3e0984679c66d07 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:33:53.157809 22581 raft_consensus.cc:504] T 00000000000000000000000000000000 P 48bfb21da5794ad3b3e0984679c66d07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48bfb21da5794ad3b3e0984679c66d07" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 40825 } }
I20260926 16:33:53.158171 22581 leader_election.cc:302] T 00000000000000000000000000000000 P 48bfb21da5794ad3b3e0984679c66d07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48bfb21da5794ad3b3e0984679c66d07; no voters:
I20260926 16:33:53.158560 22581 leader_election.cc:288] T 00000000000000000000000000000000 P 48bfb21da5794ad3b3e0984679c66d07 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:33:53.158661 22586 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 48bfb21da5794ad3b3e0984679c66d07 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:33:53.158908 22586 raft_consensus.cc:686] T 00000000000000000000000000000000 P 48bfb21da5794ad3b3e0984679c66d07 [term 1 LEADER]: Becoming Leader. State: Replica: 48bfb21da5794ad3b3e0984679c66d07, State: Running, Role: LEADER
I20260926 16:33:53.159240 22586 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48bfb21da5794ad3b3e0984679c66d07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48bfb21da5794ad3b3e0984679c66d07" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 40825 } }
I20260926 16:33:53.159741 22581 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48bfb21da5794ad3b3e0984679c66d07 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:33:53.160265 22586 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48bfb21da5794ad3b3e0984679c66d07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48bfb21da5794ad3b3e0984679c66d07. Latest consensus state: current_term: 1 leader_uuid: "48bfb21da5794ad3b3e0984679c66d07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48bfb21da5794ad3b3e0984679c66d07" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 40825 } } }
I20260926 16:33:53.160499 22586 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48bfb21da5794ad3b3e0984679c66d07 [sys.catalog]: This master's current role is: LEADER
I20260926 16:33:53.160737 22588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48bfb21da5794ad3b3e0984679c66d07 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "48bfb21da5794ad3b3e0984679c66d07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48bfb21da5794ad3b3e0984679c66d07" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 40825 } } }
I20260926 16:33:53.160876 22588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48bfb21da5794ad3b3e0984679c66d07 [sys.catalog]: This master's current role is: LEADER
I20260926 16:33:53.160959 22595 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:33:53.164463 22595 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:33:53.164856 22581 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260926 16:33:53.176016 22595 catalog_manager.cc:1347] Generated new cluster ID: 89594f1f673742f7af0b6de741dea955
I20260926 16:33:53.176095 22595 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:33:53.231398 22595 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:33:53.232865 22595 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:33:53.241616 22595 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 48bfb21da5794ad3b3e0984679c66d07: Generated new TSK 0
I20260926 16:33:53.242534 22595 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.8.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab.
Entry for principal kudu/127.11.8.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab.
I20260926 16:33:53.905864 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.8.212:40681/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.193:0
--local_ip_for_outbound_sockets=127.11.8.193
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:40825
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:44825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:33:54.023528 22626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:33:54.023821 22626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:33:54.023955 22626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:33:54.028092 22626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:33:54.028185 22626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:33:54.028277 22626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.193
I20260926 16:33:54.033995 22626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:44825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.11.8.212:40681/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:40825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.193
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:33:54.035166 22626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:33:54.036239 22626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:33:54.047676 22632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:33:54.047694 22634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:33:54.047655 22631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:33:54.048398 22626 server_base.cc:1034] running on GCE node
I20260926 16:33:54.048893 22626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:33:54.049681 22626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:33:54.050917 22626 hybrid_clock.cc:648] HybridClock initialized: now 1790440434050890 us; error 48 us; skew 500 ppm
Sep 26 16:33:54 dist-test-slave-2zvq krb5kdc[22069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440434, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:33:54.060715 22626 init.cc:377] Logged in from keytab as kudu/127.11.8.193@KRBTEST.COM (short username kudu)
I20260926 16:33:54.061954 22626 webserver.cc:460] Webserver started at http://127.11.8.193:45639/ using document root <none> and password file <none>
I20260926 16:33:54.062731 22626 fs_manager.cc:359] Metadata directory not provided
I20260926 16:33:54.062834 22626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:33:54.063073 22626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 16:33:54 dist-test-slave-2zvq krb5kdc[22069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440434, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for HTTP/127.11.8.212@KRBTEST.COM
I20260926 16:33:54.080747 22626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-0/data/instance:
uuid: "480986d25246405d974ac063780576e0"
format_stamp: "Formatted at 2026-09-26 16:33:54 on dist-test-slave-2zvq"
server_key: "429b01694581ed6aabd16ddb611e734b"
server_key_iv: "6eae2ea71b0f8f1beca74e0fb793e06c"
server_key_version: "kuduclusterkey@0"
I20260926 16:33:54.081635 22626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance:
uuid: "480986d25246405d974ac063780576e0"
format_stamp: "Formatted at 2026-09-26 16:33:54 on dist-test-slave-2zvq"
server_key: "429b01694581ed6aabd16ddb611e734b"
server_key_iv: "6eae2ea71b0f8f1beca74e0fb793e06c"
server_key_version: "kuduclusterkey@0"
I20260926 16:33:54.087100 22626 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260926 16:33:54.118697 22643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:33:54.121726 22626 fs_manager.cc:725] Time spent opening block manager: real 0.017s user 0.003s sys 0.000s
I20260926 16:33:54.121873 22626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-0/wal
uuid: "480986d25246405d974ac063780576e0"
format_stamp: "Formatted at 2026-09-26 16:33:54 on dist-test-slave-2zvq"
server_key: "429b01694581ed6aabd16ddb611e734b"
server_key_iv: "6eae2ea71b0f8f1beca74e0fb793e06c"
server_key_version: "kuduclusterkey@0"
I20260926 16:33:54.122052 22626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:33:54.145031 22626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:33:54.148627 22626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:33:54.148867 22626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:33:54.149642 22626 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:33:54.150692 22626 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:33:54.150810 22626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:33:54.150938 22626 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:33:54.151028 22626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:33:54.163125 22626 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.193:37021
I20260926 16:33:54.164273 22626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
I20260926 16:33:54.167654 22756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.193:37021 every 8 connection(s)
I20260926 16:33:54.173864 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 22626
I20260926 16:33:54.173995 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance
I20260926 16:33:54.174335 11299 external_mini_cluster.cc:1428] Setting key 68b12b436fabc74081fb47f14b345961
Sep 26 16:33:54 dist-test-slave-2zvq krb5kdc[22069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440434, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:33:54.189284 22759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:54.170482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:52303 (local address 127.11.8.254:40825)
0926 16:33:54.170778 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:54.170787 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:54.171629 (+ 842us) server_negotiation.cc:407] Connection header received
0926 16:33:54.172844 (+ 1215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:54.172848 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:54.172907 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:54.173018 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:33:54.174577 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:54.175453 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:54.179133 (+ 3680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:54.179519 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:54.179602 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:54.182785 (+ 3183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:33:54.182791 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:33:54.182794 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:33:54.182824 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:33:54.184790 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:33:54.185439 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:33:54.185443 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:33:54.185445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:33:54.185535 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:33:54.187628 (+ 2093us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:33:54.187632 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:33:54.187634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:33:54.187870 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:33:54.187983 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0926 16:33:54.188843 (+ 860us) server_negotiation.cc:299] Negotiation successful
0926 16:33:54.189002 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":91,"threads_started":1}
I20260926 16:33:54.191247 22761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:54.170950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:40825 (local address 127.11.8.193:52303)
0926 16:33:54.171471 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:54.171515 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 16:33:54.172612 (+ 1097us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:33:54.173344 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:33:54.173351 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:33:54.173677 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:33:54.174397 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:54.174421 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:54.175603 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:54.175608 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:54.178976 (+ 3368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:54.178989 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:54.179810 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:54.179814 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:54.180021 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:54.180728 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:33:54.180750 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:33:54.182561 (+ 1811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:33:54.184937 (+ 2376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:33:54.184942 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:33:54.184961 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:33:54.185305 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:33:54.185648 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:33:54.185651 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:33:54.185653 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:33:54.185838 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:33:54.188057 (+ 2219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:33:54.188065 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:33:54.188458 (+ 393us) client_negotiation.cc:769] Sending connection context
0926 16:33:54.189785 (+ 1327us) client_negotiation.cc:240] Negotiation successful
0926 16:33:54.190145 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":139,"threads_started":1}
WARNING: no policy specified for kudu/127.11.8.194@KRBTEST.COM; defaulting to no policy
I20260926 16:33:54.198031 22757 heartbeater.cc:347] Connected to a master server at 127.11.8.254:40825
I20260926 16:33:54.198315 22757 heartbeater.cc:464] Registering TS with master...
I20260926 16:33:54.198884 22757 heartbeater.cc:511] Master 127.11.8.254:40825 requested a full tablet report, sending...
I20260926 16:33:54.200460 22545 ts_manager.cc:194] Registered new tserver with Master: 480986d25246405d974ac063780576e0 (127.11.8.193:37021)
I20260926 16:33:54.201817 22545 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.193@KRBTEST.COM'} at 127.11.8.193:52303
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab.
Entry for principal kudu/127.11.8.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab.
I20260926 16:33:54.217720 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.8.212:40681/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.194:0
--local_ip_for_outbound_sockets=127.11.8.194
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:40825
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:44825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 16:33:54 dist-test-slave-2zvq krb5kdc[22069](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790440434, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260926 16:33:54.343295 22765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:33:54.343567 22765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:33:54.343643 22765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:33:54.347918 22765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:33:54.348027 22765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:33:54.348136 22765 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.194
I20260926 16:33:54.355967 22765 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:44825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.11.8.212:40681/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:40825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.194
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:33:54.357652 22765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:33:54.359030 22765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:33:54.371177 22772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:33:54.371333 22774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:33:54.371194 22771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:33:54.372381 22765 server_base.cc:1034] running on GCE node
I20260926 16:33:54.372855 22765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:33:54.373785 22765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:33:54.374976 22765 hybrid_clock.cc:648] HybridClock initialized: now 1790440434374950 us; error 47 us; skew 500 ppm
Sep 26 16:33:54 dist-test-slave-2zvq krb5kdc[22069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440434, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:33:54.378547 22765 init.cc:377] Logged in from keytab as kudu/127.11.8.194@KRBTEST.COM (short username kudu)
I20260926 16:33:54.379696 22765 webserver.cc:460] Webserver started at http://127.11.8.194:38297/ using document root <none> and password file <none>
I20260926 16:33:54.380461 22765 fs_manager.cc:359] Metadata directory not provided
I20260926 16:33:54.380589 22765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:33:54.380957 22765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 16:33:54 dist-test-slave-2zvq krb5kdc[22069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440434, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for HTTP/127.11.8.212@KRBTEST.COM
I20260926 16:33:54.396840 22765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-1/data/instance:
uuid: "155733b02ff646b49a2c1efe6713f141"
format_stamp: "Formatted at 2026-09-26 16:33:54 on dist-test-slave-2zvq"
server_key: "695015dbb1938e264528ca623954565d"
server_key_iv: "2c1db9d903fe6b423587387b80c80875"
server_key_version: "kuduclusterkey@0"
I20260926 16:33:54.397569 22765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance:
uuid: "155733b02ff646b49a2c1efe6713f141"
format_stamp: "Formatted at 2026-09-26 16:33:54 on dist-test-slave-2zvq"
server_key: "695015dbb1938e264528ca623954565d"
server_key_iv: "2c1db9d903fe6b423587387b80c80875"
server_key_version: "kuduclusterkey@0"
I20260926 16:33:54.408751 22765 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.004s sys 0.000s
I20260926 16:33:54.441579 22781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:33:54.443660 22765 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.002s sys 0.000s
I20260926 16:33:54.443804 22765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-1/wal
uuid: "155733b02ff646b49a2c1efe6713f141"
format_stamp: "Formatted at 2026-09-26 16:33:54 on dist-test-slave-2zvq"
server_key: "695015dbb1938e264528ca623954565d"
server_key_iv: "2c1db9d903fe6b423587387b80c80875"
server_key_version: "kuduclusterkey@0"
I20260926 16:33:54.443961 22765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:33:54.488421 22765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:33:54.492298 22765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:33:54.492600 22765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:33:54.493371 22765 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:33:54.494485 22765 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:33:54.494613 22765 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:33:54.494722 22765 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:33:54.494791 22765 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:33:54.507215 22765 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.194:38687
I20260926 16:33:54.508399 22765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
I20260926 16:33:54.509517 22894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.194:38687 every 8 connection(s)
I20260926 16:33:54.516247 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 22765
I20260926 16:33:54.516378 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance
I20260926 16:33:54.516701 11299 external_mini_cluster.cc:1428] Setting key 437a3ff19bb9a40c6f02e048137e7c77
Sep 26 16:33:54 dist-test-slave-2zvq krb5kdc[22069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440434, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.8.195@KRBTEST.COM; defaulting to no policy
I20260926 16:33:54.533630 22759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:54.512314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:39571 (local address 127.11.8.254:40825)
0926 16:33:54.512455 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:54.512460 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:54.513653 (+ 1193us) server_negotiation.cc:407] Connection header received
0926 16:33:54.516148 (+ 2495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:54.516152 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:54.516213 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:54.516272 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:33:54.518338 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:54.519215 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:54.524198 (+ 4983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:54.524551 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:54.524638 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:54.529765 (+ 5127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:33:54.529770 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:33:54.529773 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:33:54.529804 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:33:54.531437 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:33:54.531998 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:33:54.532001 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:33:54.532002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:33:54.532082 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:33:54.532449 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:33:54.532452 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:33:54.532454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:33:54.532700 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:33:54.532805 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 16:33:54.533329 (+ 524us) server_negotiation.cc:299] Negotiation successful
0926 16:33:54.533464 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 16:33:54.534592 22897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:54.512764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:40825 (local address 127.11.8.194:39571)
0926 16:33:54.513482 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:54.513544 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0926 16:33:54.515867 (+ 2323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:33:54.516698 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:33:54.516709 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:33:54.517091 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:33:54.518130 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:54.518154 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:54.520863 (+ 2709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:54.520867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:54.522480 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:54.522493 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:54.524862 (+ 2369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:54.524867 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:54.525096 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:54.527688 (+ 2592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:33:54.527712 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:33:54.529504 (+ 1792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:33:54.531576 (+ 2072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:33:54.531582 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:33:54.531601 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:33:54.531875 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:33:54.532181 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:33:54.532184 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:33:54.532186 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:33:54.532344 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:33:54.532802 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:33:54.532810 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:33:54.533083 (+ 273us) client_negotiation.cc:769] Sending connection context
0926 16:33:54.533279 (+ 196us) client_negotiation.cc:240] Negotiation successful
0926 16:33:54.533626 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":140,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.195@KRBTEST.COM" created.
I20260926 16:33:54.535769 22895 heartbeater.cc:347] Connected to a master server at 127.11.8.254:40825
I20260926 16:33:54.536031 22895 heartbeater.cc:464] Registering TS with master...
I20260926 16:33:54.536553 22895 heartbeater.cc:511] Master 127.11.8.254:40825 requested a full tablet report, sending...
I20260926 16:33:54.537587 22545 ts_manager.cc:194] Registered new tserver with Master: 155733b02ff646b49a2c1efe6713f141 (127.11.8.194:38687)
I20260926 16:33:54.538187 22545 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.194@KRBTEST.COM'} at 127.11.8.194:39571
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab.
Entry for principal kudu/127.11.8.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab.
I20260926 16:33:54.558496 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.8.212:40681/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.195:0
--local_ip_for_outbound_sockets=127.11.8.195
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:40825
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:44825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:33:54.718561 22900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:33:54.718976 22900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:33:54.719116 22900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
Sep 26 16:33:54 dist-test-slave-2zvq krb5kdc[22069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440434, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for HTTP/127.11.8.212@KRBTEST.COM
W20260926 16:33:54.724934 22900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:33:54.725076 22900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:33:54.725217 22900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.195
I20260926 16:33:54.732642 22900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:44825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.11.8.212:40681/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:40825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.195
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:33:54.734210 22900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:33:54.735778 22900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:33:54.748433 22906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:33:54.748361 22908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:33:54.748497 22900 server_base.cc:1034] running on GCE node
W20260926 16:33:54.748548 22905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:33:54.749557 22900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:33:54.750355 22900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:33:54.751513 22900 hybrid_clock.cc:648] HybridClock initialized: now 1790440434751445 us; error 81 us; skew 500 ppm
Sep 26 16:33:54 dist-test-slave-2zvq krb5kdc[22069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440434, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:33:54.755203 22900 init.cc:377] Logged in from keytab as kudu/127.11.8.195@KRBTEST.COM (short username kudu)
I20260926 16:33:54.756347 22900 webserver.cc:460] Webserver started at http://127.11.8.195:33811/ using document root <none> and password file <none>
I20260926 16:33:54.757196 22900 fs_manager.cc:359] Metadata directory not provided
I20260926 16:33:54.757328 22900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:33:54.757604 22900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 16:33:54 dist-test-slave-2zvq krb5kdc[22069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440434, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for HTTP/127.11.8.212@KRBTEST.COM
I20260926 16:33:54.782590 22900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-2/data/instance:
uuid: "aeaa85ea3e4844c3a4ae5d84778b8a73"
format_stamp: "Formatted at 2026-09-26 16:33:54 on dist-test-slave-2zvq"
server_key: "df1af96acb0df56fea2f0f4fdc534034"
server_key_iv: "2d7e60fc568051b92a109bf257c38bce"
server_key_version: "kuduclusterkey@0"
I20260926 16:33:54.783304 22900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance:
uuid: "aeaa85ea3e4844c3a4ae5d84778b8a73"
format_stamp: "Formatted at 2026-09-26 16:33:54 on dist-test-slave-2zvq"
server_key: "df1af96acb0df56fea2f0f4fdc534034"
server_key_iv: "2d7e60fc568051b92a109bf257c38bce"
server_key_version: "kuduclusterkey@0"
I20260926 16:33:54.792961 22900 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.005s sys 0.000s
I20260926 16:33:54.814483 22915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:33:54.816277 22900 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.002s sys 0.001s
I20260926 16:33:54.816413 22900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-2/wal
uuid: "aeaa85ea3e4844c3a4ae5d84778b8a73"
format_stamp: "Formatted at 2026-09-26 16:33:54 on dist-test-slave-2zvq"
server_key: "df1af96acb0df56fea2f0f4fdc534034"
server_key_iv: "2d7e60fc568051b92a109bf257c38bce"
server_key_version: "kuduclusterkey@0"
I20260926 16:33:54.816550 22900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:33:54.839843 22900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:33:54.843072 22900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:33:54.843282 22900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:33:54.843946 22900 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:33:54.844844 22900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:33:54.844915 22900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:33:54.845016 22900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:33:54.845072 22900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:33:54.855743 22900 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.195:33411
I20260926 16:33:54.855768 23028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.195:33411 every 8 connection(s)
I20260926 16:33:54.856757 22900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
I20260926 16:33:54.858605 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 22900
I20260926 16:33:54.858729 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance
I20260926 16:33:54.859054 11299 external_mini_cluster.cc:1428] Setting key f530d340e127df45c0052565f6796a1e
Sep 26 16:33:54 dist-test-slave-2zvq krb5kdc[22069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440434, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:33:54.876013 22759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:54.859441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:43787 (local address 127.11.8.254:40825)
0926 16:33:54.859614 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:54.859619 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:54.863495 (+ 3876us) server_negotiation.cc:407] Connection header received
0926 16:33:54.864601 (+ 1106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:54.864605 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:54.864656 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:54.864746 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:33:54.866114 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:54.866946 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:54.868535 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:54.868877 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:54.868992 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:54.871980 (+ 2988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:33:54.871984 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:33:54.871987 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:33:54.872013 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:33:54.873674 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:33:54.874299 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:33:54.874303 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:33:54.874305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:33:54.874387 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:33:54.874781 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:33:54.874785 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:33:54.874787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:33:54.875003 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:33:54.875074 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 16:33:54.875723 (+ 649us) server_negotiation.cc:299] Negotiation successful
0926 16:33:54.875840 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260926 16:33:54.877518 23031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:54.860437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:40825 (local address 127.11.8.195:43787)
0926 16:33:54.863327 (+ 2890us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:54.863378 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 16:33:54.864363 (+ 985us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:33:54.864961 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:33:54.864971 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:33:54.865308 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:33:54.865943 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:54.865960 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:54.867087 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:54.867090 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:54.868398 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:54.868409 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:54.869044 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:54.869047 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:54.869229 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:54.870075 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:33:54.870096 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:33:54.871779 (+ 1683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:33:54.873798 (+ 2019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:33:54.873803 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:33:54.873819 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:33:54.874156 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:33:54.874500 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:33:54.874503 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:33:54.874505 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:33:54.874675 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:33:54.875156 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:33:54.875162 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:33:54.875457 (+ 295us) client_negotiation.cc:769] Sending connection context
0926 16:33:54.876168 (+ 711us) client_negotiation.cc:240] Negotiation successful
0926 16:33:54.876480 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2662,"thread_start_us":127,"threads_started":1}
I20260926 16:33:54.879065 23029 heartbeater.cc:347] Connected to a master server at 127.11.8.254:40825
I20260926 16:33:54.879331 23029 heartbeater.cc:464] Registering TS with master...
I20260926 16:33:54.879905 23029 heartbeater.cc:511] Master 127.11.8.254:40825 requested a full tablet report, sending...
I20260926 16:33:54.881158 22545 ts_manager.cc:194] Registered new tserver with Master: aeaa85ea3e4844c3a4ae5d84778b8a73 (127.11.8.195:33411)
I20260926 16:33:54.881877 22545 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.195@KRBTEST.COM'} at 127.11.8.195:43787
I20260926 16:33:54.885391 11299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 16:33:54.896670 22759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:54.887498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40644 (local address 127.11.8.254:40825)
0926 16:33:54.887622 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:54.887626 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:54.887929 (+ 303us) server_negotiation.cc:407] Connection header received
0926 16:33:54.888037 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:54.888040 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:54.888094 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:54.888180 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:33:54.889130 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:54.889936 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:54.891105 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:54.891446 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:54.891560 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:54.893185 (+ 1625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:33:54.893188 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:33:54.893192 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:33:54.893217 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:33:54.894997 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:33:54.895436 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:33:54.895440 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:33:54.895442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:33:54.895515 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:33:54.895772 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:33:54.895775 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:33:54.895777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:33:54.895970 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:33:54.896063 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 16:33:54.896404 (+ 341us) server_negotiation.cc:299] Negotiation successful
0926 16:33:54.896521 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 16:33:54.900492 22545 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40644:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:33:54.905205 22545 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:33:54.925197 23041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:54.916593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:38687 (local address 127.0.0.1:47878)
0926 16:33:54.917309 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:54.917364 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0926 16:33:54.917548 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:33:54.918363 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:33:54.918369 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:33:54.919707 (+ 1338us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:33:54.919865 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:54.919873 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:54.921606 (+ 1733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:54.921610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:54.923266 (+ 1656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:54.923277 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:54.924636 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:54.924639 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:54.924752 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:54.924796 (+ 44us) client_negotiation.cc:769] Sending connection context
0926 16:33:54.924916 (+ 120us) client_negotiation.cc:240] Negotiation successful
0926 16:33:54.925040 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":109,"threads_started":1}
I20260926 16:33:54.926090 23042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:54.916767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47878 (local address 127.11.8.194:38687)
0926 16:33:54.917822 (+ 1055us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:54.917833 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:54.917855 (+ 22us) server_negotiation.cc:407] Connection header received
0926 16:33:54.917935 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:54.917940 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:54.918157 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:54.920037 (+ 1880us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:33:54.920174 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:54.921469 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:54.923442 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:54.924290 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:54.924412 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:54.924568 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0926 16:33:54.925840 (+ 1272us) server_negotiation.cc:299] Negotiation successful
0926 16:33:54.925947 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":956,"thread_start_us":118,"threads_started":1}
I20260926 16:33:54.926218 23045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:54.918955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:33411 (local address 127.0.0.1:56464)
0926 16:33:54.919160 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:54.919171 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 16:33:54.919236 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:33:54.920966 (+ 1730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:33:54.920969 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:33:54.920985 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:33:54.921087 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:54.921093 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:54.922552 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:54.922572 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:54.924634 (+ 2062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:54.924644 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:54.925833 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:54.925837 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:54.925929 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:54.925953 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 16:33:54.926002 (+ 49us) client_negotiation.cc:240] Negotiation successful
0926 16:33:54.926079 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":87,"threads_started":1}
I20260926 16:33:54.926828 23046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:54.919358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56464 (local address 127.11.8.195:33411)
0926 16:33:54.919738 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:54.919749 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:54.919771 (+ 22us) server_negotiation.cc:407] Connection header received
0926 16:33:54.919843 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:54.919847 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:54.920826 (+ 979us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:54.921216 (+ 390us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:33:54.921336 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:54.922424 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:54.924800 (+ 2376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:54.925696 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:54.926416 (+ 720us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:54.926560 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0926 16:33:54.926610 (+ 50us) server_negotiation.cc:299] Negotiation successful
0926 16:33:54.926695 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":160,"threads_started":1}
I20260926 16:33:54.930380 22946 tablet_service.cc:1467] Processing CreateTablet for tablet 465042b990654a7491ba52031defabac (DEFAULT_TABLE table=test-table [id=eb9ee71c94d9447fb805adf1e00ba5ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:33:54.930524 22829 tablet_service.cc:1467] Processing CreateTablet for tablet 465042b990654a7491ba52031defabac (DEFAULT_TABLE table=test-table [id=eb9ee71c94d9447fb805adf1e00ba5ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:33:54.931488 22946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 465042b990654a7491ba52031defabac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:33:54.931531 22829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 465042b990654a7491ba52031defabac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:33:54.933594 23043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:54.918597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:37021 (local address 127.0.0.1:34684)
0926 16:33:54.918825 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:54.918839 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:33:54.918939 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:33:54.929492 (+ 10553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:33:54.929495 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:33:54.929525 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:33:54.929621 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:54.929626 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:54.931085 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:54.931089 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:54.932172 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:54.932182 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:54.933214 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:54.933217 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:54.933314 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:54.933340 (+ 26us) client_negotiation.cc:769] Sending connection context
0926 16:33:54.933388 (+ 48us) client_negotiation.cc:240] Negotiation successful
0926 16:33:54.933451 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":100,"threads_started":1}
I20260926 16:33:54.934464 23044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:54.918751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34684 (local address 127.11.8.193:37021)
0926 16:33:54.929047 (+ 10296us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:54.929056 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:54.929080 (+ 24us) server_negotiation.cc:407] Connection header received
0926 16:33:54.929153 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:54.929157 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:54.929343 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:54.929769 (+ 426us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:33:54.929906 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:54.930955 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:54.932312 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:54.933079 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:54.934016 (+ 937us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:54.934157 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0926 16:33:54.934208 (+ 51us) server_negotiation.cc:299] Negotiation successful
0926 16:33:54.934319 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10186,"thread_start_us":124,"threads_started":1}
I20260926 16:33:54.937546 22688 tablet_service.cc:1467] Processing CreateTablet for tablet 465042b990654a7491ba52031defabac (DEFAULT_TABLE table=test-table [id=eb9ee71c94d9447fb805adf1e00ba5ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:33:54.938673 22688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 465042b990654a7491ba52031defabac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:33:54.940828 23047 tablet_bootstrap.cc:492] T 465042b990654a7491ba52031defabac P 155733b02ff646b49a2c1efe6713f141: Bootstrap starting.
I20260926 16:33:54.943215 23047 tablet_bootstrap.cc:654] T 465042b990654a7491ba52031defabac P 155733b02ff646b49a2c1efe6713f141: Neither blocks nor log segments found. Creating new log.
I20260926 16:33:54.944671 23047 log.cc:824] T 465042b990654a7491ba52031defabac P 155733b02ff646b49a2c1efe6713f141: Log is configured to *not* fsync() on all Append() calls
I20260926 16:33:54.945765 23049 tablet_bootstrap.cc:492] T 465042b990654a7491ba52031defabac P 480986d25246405d974ac063780576e0: Bootstrap starting.
I20260926 16:33:54.946789 23047 tablet_bootstrap.cc:492] T 465042b990654a7491ba52031defabac P 155733b02ff646b49a2c1efe6713f141: No bootstrap required, opened a new log
I20260926 16:33:54.947006 23047 ts_tablet_manager.cc:1397] T 465042b990654a7491ba52031defabac P 155733b02ff646b49a2c1efe6713f141: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260926 16:33:54.948078 23049 tablet_bootstrap.cc:654] T 465042b990654a7491ba52031defabac P 480986d25246405d974ac063780576e0: Neither blocks nor log segments found. Creating new log.
I20260926 16:33:54.949499 23049 log.cc:824] T 465042b990654a7491ba52031defabac P 480986d25246405d974ac063780576e0: Log is configured to *not* fsync() on all Append() calls
I20260926 16:33:54.950829 23047 raft_consensus.cc:348] T 465042b990654a7491ba52031defabac P 155733b02ff646b49a2c1efe6713f141 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "155733b02ff646b49a2c1efe6713f141" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 38687 } } peers { permanent_uuid: "480986d25246405d974ac063780576e0" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 37021 } } peers { permanent_uuid: "aeaa85ea3e4844c3a4ae5d84778b8a73" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33411 } }
I20260926 16:33:54.951212 23047 raft_consensus.cc:374] T 465042b990654a7491ba52031defabac P 155733b02ff646b49a2c1efe6713f141 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:33:54.951303 23047 raft_consensus.cc:729] T 465042b990654a7491ba52031defabac P 155733b02ff646b49a2c1efe6713f141 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 155733b02ff646b49a2c1efe6713f141, State: Initialized, Role: FOLLOWER
I20260926 16:33:54.951709 23049 tablet_bootstrap.cc:492] T 465042b990654a7491ba52031defabac P 480986d25246405d974ac063780576e0: No bootstrap required, opened a new log
I20260926 16:33:54.951906 23049 ts_tablet_manager.cc:1397] T 465042b990654a7491ba52031defabac P 480986d25246405d974ac063780576e0: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 16:33:54.951891 23047 consensus_queue.cc:260] T 465042b990654a7491ba52031defabac P 155733b02ff646b49a2c1efe6713f141 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "155733b02ff646b49a2c1efe6713f141" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 38687 } } peers { permanent_uuid: "480986d25246405d974ac063780576e0" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 37021 } } peers { permanent_uuid: "aeaa85ea3e4844c3a4ae5d84778b8a73" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33411 } }
I20260926 16:33:54.952976 23047 ts_tablet_manager.cc:1428] T 465042b990654a7491ba52031defabac P 155733b02ff646b49a2c1efe6713f141: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260926 16:33:54.953356 22895 heartbeater.cc:503] Master 127.11.8.254:40825 was elected leader, sending a full tablet report...
I20260926 16:33:54.953917 23048 tablet_bootstrap.cc:492] T 465042b990654a7491ba52031defabac P aeaa85ea3e4844c3a4ae5d84778b8a73: Bootstrap starting.
I20260926 16:33:54.956115 23049 raft_consensus.cc:348] T 465042b990654a7491ba52031defabac P 480986d25246405d974ac063780576e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "155733b02ff646b49a2c1efe6713f141" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 38687 } } peers { permanent_uuid: "480986d25246405d974ac063780576e0" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 37021 } } peers { permanent_uuid: "aeaa85ea3e4844c3a4ae5d84778b8a73" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33411 } }
I20260926 16:33:54.956413 23049 raft_consensus.cc:374] T 465042b990654a7491ba52031defabac P 480986d25246405d974ac063780576e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:33:54.956496 23049 raft_consensus.cc:729] T 465042b990654a7491ba52031defabac P 480986d25246405d974ac063780576e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 480986d25246405d974ac063780576e0, State: Initialized, Role: FOLLOWER
I20260926 16:33:54.956679 23048 tablet_bootstrap.cc:654] T 465042b990654a7491ba52031defabac P aeaa85ea3e4844c3a4ae5d84778b8a73: Neither blocks nor log segments found. Creating new log.
I20260926 16:33:54.957052 23049 consensus_queue.cc:260] T 465042b990654a7491ba52031defabac P 480986d25246405d974ac063780576e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "155733b02ff646b49a2c1efe6713f141" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 38687 } } peers { permanent_uuid: "480986d25246405d974ac063780576e0" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 37021 } } peers { permanent_uuid: "aeaa85ea3e4844c3a4ae5d84778b8a73" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33411 } }
I20260926 16:33:54.957466 23048 log.cc:824] T 465042b990654a7491ba52031defabac P aeaa85ea3e4844c3a4ae5d84778b8a73: Log is configured to *not* fsync() on all Append() calls
I20260926 16:33:54.958073 23049 ts_tablet_manager.cc:1428] T 465042b990654a7491ba52031defabac P 480986d25246405d974ac063780576e0: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260926 16:33:54.958434 22757 heartbeater.cc:503] Master 127.11.8.254:40825 was elected leader, sending a full tablet report...
I20260926 16:33:54.962397 23048 tablet_bootstrap.cc:492] T 465042b990654a7491ba52031defabac P aeaa85ea3e4844c3a4ae5d84778b8a73: No bootstrap required, opened a new log
I20260926 16:33:54.962648 23048 ts_tablet_manager.cc:1397] T 465042b990654a7491ba52031defabac P aeaa85ea3e4844c3a4ae5d84778b8a73: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260926 16:33:54.966619 23048 raft_consensus.cc:348] T 465042b990654a7491ba52031defabac P aeaa85ea3e4844c3a4ae5d84778b8a73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "155733b02ff646b49a2c1efe6713f141" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 38687 } } peers { permanent_uuid: "480986d25246405d974ac063780576e0" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 37021 } } peers { permanent_uuid: "aeaa85ea3e4844c3a4ae5d84778b8a73" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33411 } }
I20260926 16:33:54.966953 23048 raft_consensus.cc:374] T 465042b990654a7491ba52031defabac P aeaa85ea3e4844c3a4ae5d84778b8a73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:33:54.967052 23048 raft_consensus.cc:729] T 465042b990654a7491ba52031defabac P aeaa85ea3e4844c3a4ae5d84778b8a73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aeaa85ea3e4844c3a4ae5d84778b8a73, State: Initialized, Role: FOLLOWER
I20260926 16:33:54.967597 23048 consensus_queue.cc:260] T 465042b990654a7491ba52031defabac P aeaa85ea3e4844c3a4ae5d84778b8a73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "155733b02ff646b49a2c1efe6713f141" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 38687 } } peers { permanent_uuid: "480986d25246405d974ac063780576e0" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 37021 } } peers { permanent_uuid: "aeaa85ea3e4844c3a4ae5d84778b8a73" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33411 } }
I20260926 16:33:54.968359 23029 heartbeater.cc:503] Master 127.11.8.254:40825 was elected leader, sending a full tablet report...
I20260926 16:33:54.968665 23048 ts_tablet_manager.cc:1428] T 465042b990654a7491ba52031defabac P aeaa85ea3e4844c3a4ae5d84778b8a73: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
Sep 26 16:33:55 dist-test-slave-2zvq krb5kdc[22069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440434, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for HTTP/127.11.8.212@KRBTEST.COM
W20260926 16:33:55.016697 22896 tablet.cc:2369] T 465042b990654a7491ba52031defabac P 155733b02ff646b49a2c1efe6713f141: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 16:33:55.113381 23030 tablet.cc:2369] T 465042b990654a7491ba52031defabac P aeaa85ea3e4844c3a4ae5d84778b8a73: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 16:33:55.170924 22758 tablet.cc:2369] T 465042b990654a7491ba52031defabac P 480986d25246405d974ac063780576e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:33:55.190526 23052 raft_consensus.cc:482] T 465042b990654a7491ba52031defabac P 155733b02ff646b49a2c1efe6713f141 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:33:55.190757 23052 raft_consensus.cc:504] T 465042b990654a7491ba52031defabac P 155733b02ff646b49a2c1efe6713f141 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "155733b02ff646b49a2c1efe6713f141" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 38687 } } peers { permanent_uuid: "480986d25246405d974ac063780576e0" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 37021 } } peers { permanent_uuid: "aeaa85ea3e4844c3a4ae5d84778b8a73" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33411 } }
I20260926 16:33:55.192080 23052 leader_election.cc:288] T 465042b990654a7491ba52031defabac P 155733b02ff646b49a2c1efe6713f141 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 480986d25246405d974ac063780576e0 (127.11.8.193:37021), aeaa85ea3e4844c3a4ae5d84778b8a73 (127.11.8.195:33411)
I20260926 16:33:55.197106 23044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:55.192445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:49569 (local address 127.11.8.193:37021)
0926 16:33:55.192608 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:55.192614 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:55.192767 (+ 153us) server_negotiation.cc:407] Connection header received
0926 16:33:55.192973 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:55.192977 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:55.193026 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:55.193073 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:33:55.193467 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:55.194237 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:55.195822 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:55.196406 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:55.196513 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:55.196570 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0926 16:33:55.196906 (+ 336us) server_negotiation.cc:299] Negotiation successful
0926 16:33:55.196979 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 16:33:55.197500 23046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:55.193027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:50977 (local address 127.11.8.195:33411)
0926 16:33:55.193138 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:55.193143 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:55.193272 (+ 129us) server_negotiation.cc:407] Connection header received
0926 16:33:55.193443 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:55.193446 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:55.193492 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:55.193552 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:33:55.193920 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:55.194721 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:55.196227 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:55.196844 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:55.196934 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:55.196978 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0926 16:33:55.197317 (+ 339us) server_negotiation.cc:299] Negotiation successful
0926 16:33:55.197383 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 16:33:55.197932 23059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:55.192688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:33411 (local address 127.11.8.194:50977)
0926 16:33:55.193194 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:55.193207 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:33:55.193334 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:33:55.193678 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:33:55.193681 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:33:55.193696 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:33:55.193802 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:55.193807 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:55.194877 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:55.194880 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:55.196090 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:55.196100 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:55.197074 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:55.197077 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:55.197189 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:55.197208 (+ 19us) client_negotiation.cc:769] Sending connection context
0926 16:33:55.197682 (+ 474us) client_negotiation.cc:240] Negotiation successful
0926 16:33:55.197751 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":89,"threads_started":1}
I20260926 16:33:55.197954 23058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:55.192318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:37021 (local address 127.11.8.194:49569)
0926 16:33:55.192669 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:55.192684 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:33:55.192848 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:33:55.193204 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:33:55.193207 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:33:55.193222 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:33:55.193332 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:55.193338 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:55.194406 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:55.194409 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:55.195656 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:33:55.195669 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:33:55.196667 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:33:55.196671 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:33:55.196760 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:55.196785 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 16:33:55.197694 (+ 909us) client_negotiation.cc:240] Negotiation successful
0926 16:33:55.197774 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":4,"thread_start_us":86,"threads_started":1}
I20260926 16:33:55.198645 22982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "465042b990654a7491ba52031defabac" candidate_uuid: "155733b02ff646b49a2c1efe6713f141" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeaa85ea3e4844c3a4ae5d84778b8a73" is_pre_election: true
I20260926 16:33:55.198869 22697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "465042b990654a7491ba52031defabac" candidate_uuid: "155733b02ff646b49a2c1efe6713f141" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "480986d25246405d974ac063780576e0" is_pre_election: true
I20260926 16:33:55.198992 22982 raft_consensus.cc:2454] T 465042b990654a7491ba52031defabac P aeaa85ea3e4844c3a4ae5d84778b8a73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 155733b02ff646b49a2c1efe6713f141 in term 0.
I20260926 16:33:55.199098 22697 raft_consensus.cc:2454] T 465042b990654a7491ba52031defabac P 480986d25246405d974ac063780576e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 155733b02ff646b49a2c1efe6713f141 in term 0.
I20260926 16:33:55.199491 22782 leader_election.cc:302] T 465042b990654a7491ba52031defabac P 155733b02ff646b49a2c1efe6713f141 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 155733b02ff646b49a2c1efe6713f141, 480986d25246405d974ac063780576e0; no voters:
I20260926 16:33:55.200021 23052 raft_consensus.cc:2781] T 465042b990654a7491ba52031defabac P 155733b02ff646b49a2c1efe6713f141 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:33:55.200109 23052 raft_consensus.cc:482] T 465042b990654a7491ba52031defabac P 155733b02ff646b49a2c1efe6713f141 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:33:55.200163 23052 raft_consensus.cc:3037] T 465042b990654a7491ba52031defabac P 155733b02ff646b49a2c1efe6713f141 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:33:55.201107 23052 raft_consensus.cc:504] T 465042b990654a7491ba52031defabac P 155733b02ff646b49a2c1efe6713f141 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "155733b02ff646b49a2c1efe6713f141" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 38687 } } peers { permanent_uuid: "480986d25246405d974ac063780576e0" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 37021 } } peers { permanent_uuid: "aeaa85ea3e4844c3a4ae5d84778b8a73" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33411 } }
I20260926 16:33:55.201408 23052 leader_election.cc:288] T 465042b990654a7491ba52031defabac P 155733b02ff646b49a2c1efe6713f141 [CANDIDATE]: Term 1 election: Requested vote from peers 480986d25246405d974ac063780576e0 (127.11.8.193:37021), aeaa85ea3e4844c3a4ae5d84778b8a73 (127.11.8.195:33411)
I20260926 16:33:55.201756 22982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "465042b990654a7491ba52031defabac" candidate_uuid: "155733b02ff646b49a2c1efe6713f141" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeaa85ea3e4844c3a4ae5d84778b8a73"
I20260926 16:33:55.201749 22697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "465042b990654a7491ba52031defabac" candidate_uuid: "155733b02ff646b49a2c1efe6713f141" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "480986d25246405d974ac063780576e0"
I20260926 16:33:55.201860 22697 raft_consensus.cc:3037] T 465042b990654a7491ba52031defabac P 480986d25246405d974ac063780576e0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:33:55.201860 22982 raft_consensus.cc:3037] T 465042b990654a7491ba52031defabac P aeaa85ea3e4844c3a4ae5d84778b8a73 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:33:55.203043 22697 raft_consensus.cc:2454] T 465042b990654a7491ba52031defabac P 480986d25246405d974ac063780576e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 155733b02ff646b49a2c1efe6713f141 in term 1.
I20260926 16:33:55.203047 22982 raft_consensus.cc:2454] T 465042b990654a7491ba52031defabac P aeaa85ea3e4844c3a4ae5d84778b8a73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 155733b02ff646b49a2c1efe6713f141 in term 1.
I20260926 16:33:55.203356 22782 leader_election.cc:302] T 465042b990654a7491ba52031defabac P 155733b02ff646b49a2c1efe6713f141 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 155733b02ff646b49a2c1efe6713f141, 480986d25246405d974ac063780576e0; no voters:
I20260926 16:33:55.203519 23052 raft_consensus.cc:2781] T 465042b990654a7491ba52031defabac P 155733b02ff646b49a2c1efe6713f141 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:33:55.203754 23052 raft_consensus.cc:686] T 465042b990654a7491ba52031defabac P 155733b02ff646b49a2c1efe6713f141 [term 1 LEADER]: Becoming Leader. State: Replica: 155733b02ff646b49a2c1efe6713f141, State: Running, Role: LEADER
I20260926 16:33:55.204078 23052 consensus_queue.cc:237] T 465042b990654a7491ba52031defabac P 155733b02ff646b49a2c1efe6713f141 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "155733b02ff646b49a2c1efe6713f141" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 38687 } } peers { permanent_uuid: "480986d25246405d974ac063780576e0" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 37021 } } peers { permanent_uuid: "aeaa85ea3e4844c3a4ae5d84778b8a73" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33411 } }
I20260926 16:33:55.207523 22545 catalog_manager.cc:5495] T 465042b990654a7491ba52031defabac P 155733b02ff646b49a2c1efe6713f141 reported cstate change: term changed from 0 to 1, leader changed from <none> to 155733b02ff646b49a2c1efe6713f141 (127.11.8.194). New cstate: current_term: 1 leader_uuid: "155733b02ff646b49a2c1efe6713f141" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "155733b02ff646b49a2c1efe6713f141" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 38687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "480986d25246405d974ac063780576e0" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 37021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aeaa85ea3e4844c3a4ae5d84778b8a73" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33411 } health_report { overall_health: UNKNOWN } } }
I20260926 16:33:55.284548 23042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:55.281299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47880 (local address 127.11.8.194:38687)
0926 16:33:55.281421 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0926 16:33:55.281425 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:33:55.281453 (+ 28us) server_negotiation.cc:407] Connection header received
0926 16:33:55.281525 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:33:55.281527 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:33:55.281584 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:33:55.281667 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:33:55.281871 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:55.282435 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:55.283306 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:33:55.283510 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:33:55.283589 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:33:55.283825 (+ 236us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:33:55.284250 (+ 425us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:33:55.284339 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 16:33:55.284395 (+ 56us) server_negotiation.cc:299] Negotiation successful
0926 16:33:55.284442 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 16:33:55.290541 22697 raft_consensus.cc:1261] T 465042b990654a7491ba52031defabac P 480986d25246405d974ac063780576e0 [term 1 FOLLOWER]: Refusing update from remote peer 155733b02ff646b49a2c1efe6713f141: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:33:55.290671 22982 raft_consensus.cc:1261] T 465042b990654a7491ba52031defabac P aeaa85ea3e4844c3a4ae5d84778b8a73 [term 1 FOLLOWER]: Refusing update from remote peer 155733b02ff646b49a2c1efe6713f141: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:33:55.291335 23060 consensus_queue.cc:1035] T 465042b990654a7491ba52031defabac P 155733b02ff646b49a2c1efe6713f141 [LEADER]: Connected to new peer: Peer: permanent_uuid: "480986d25246405d974ac063780576e0" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 37021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:33:55.291536 23052 consensus_queue.cc:1035] T 465042b990654a7491ba52031defabac P 155733b02ff646b49a2c1efe6713f141 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aeaa85ea3e4844c3a4ae5d84778b8a73" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:33:55.306694 22545 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:33:55.306869 22545 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:33:55.309142 22545 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 48bfb21da5794ad3b3e0984679c66d07: Sending DeleteTablet for 3 replicas of tablet 465042b990654a7491ba52031defabac
I20260926 16:33:55.309809 22829 tablet_service.cc:1514] Processing DeleteTablet for tablet 465042b990654a7491ba52031defabac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:33:55 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:47878
I20260926 16:33:55.310132 22688 tablet_service.cc:1514] Processing DeleteTablet for tablet 465042b990654a7491ba52031defabac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:33:55 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:34684
I20260926 16:33:55.310293 23079 tablet_replica.cc:330] T 465042b990654a7491ba52031defabac P 155733b02ff646b49a2c1efe6713f141: stopping tablet replica
I20260926 16:33:55.310446 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 22626
I20260926 16:33:55.310609 22946 tablet_service.cc:1514] Processing DeleteTablet for tablet 465042b990654a7491ba52031defabac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:33:55 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:56464
I20260926 16:33:55.311025 23079 raft_consensus.cc:2229] T 465042b990654a7491ba52031defabac P 155733b02ff646b49a2c1efe6713f141 [term 1 LEADER]: Raft consensus shutting down.
I20260926 16:33:55.311273 23081 tablet_replica.cc:330] T 465042b990654a7491ba52031defabac P aeaa85ea3e4844c3a4ae5d84778b8a73: stopping tablet replica
I20260926 16:33:55.311388 23079 raft_consensus.cc:2258] T 465042b990654a7491ba52031defabac P 155733b02ff646b49a2c1efe6713f141 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:33:55.311604 23081 raft_consensus.cc:2229] T 465042b990654a7491ba52031defabac P aeaa85ea3e4844c3a4ae5d84778b8a73 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:33:55.312018 23081 raft_consensus.cc:2258] T 465042b990654a7491ba52031defabac P aeaa85ea3e4844c3a4ae5d84778b8a73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:33:55.313369 23079 ts_tablet_manager.cc:1905] T 465042b990654a7491ba52031defabac P 155733b02ff646b49a2c1efe6713f141: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:33:55.313413 23081 ts_tablet_manager.cc:1905] T 465042b990654a7491ba52031defabac P aeaa85ea3e4844c3a4ae5d84778b8a73: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:33:55.316329 23081 ts_tablet_manager.cc:1918] T 465042b990654a7491ba52031defabac P aeaa85ea3e4844c3a4ae5d84778b8a73: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:33:55.316459 23081 log.cc:1196] T 465042b990654a7491ba52031defabac P aeaa85ea3e4844c3a4ae5d84778b8a73: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-2/wal/wals/465042b990654a7491ba52031defabac
I20260926 16:33:55.316861 23081 ts_tablet_manager.cc:1939] T 465042b990654a7491ba52031defabac P aeaa85ea3e4844c3a4ae5d84778b8a73: Deleting consensus metadata
I20260926 16:33:55.317131 23079 ts_tablet_manager.cc:1918] T 465042b990654a7491ba52031defabac P 155733b02ff646b49a2c1efe6713f141: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:33:55.317286 23079 log.cc:1196] T 465042b990654a7491ba52031defabac P 155733b02ff646b49a2c1efe6713f141: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790440324823152-11299-0/minicluster-data/ts-1/wal/wals/465042b990654a7491ba52031defabac
I20260926 16:33:55.317703 23079 ts_tablet_manager.cc:1939] T 465042b990654a7491ba52031defabac P 155733b02ff646b49a2c1efe6713f141: Deleting consensus metadata
I20260926 16:33:55.318120 22529 catalog_manager.cc:4841] TS aeaa85ea3e4844c3a4ae5d84778b8a73 (127.11.8.195:33411): tablet 465042b990654a7491ba52031defabac (table test-table [id=eb9ee71c94d9447fb805adf1e00ba5ca]) successfully deleted
I20260926 16:33:55.318873 22532 catalog_manager.cc:4841] TS 155733b02ff646b49a2c1efe6713f141 (127.11.8.194:38687): tablet 465042b990654a7491ba52031defabac (table test-table [id=eb9ee71c94d9447fb805adf1e00ba5ca]) successfully deleted
W20260926 16:33:55.320039 22529 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.8.193:37021 (error 108)
I20260926 16:33:55.320989 23043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:33:55.320699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:37021 (local address 127.0.0.1:34690)
0926 16:33:55.320819 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0926 16:33:55.320890 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.193:37021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260926 16:33:55.321094 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 22765
W20260926 16:33:55.321916 22529 catalog_manager.cc:4584] TS 480986d25246405d974ac063780576e0 (127.11.8.193:37021): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 465042b990654a7491ba52031defabac: Network error: Client connection negotiation failed: client connection to 127.11.8.193:37021: connect: Connection refused (error 111)
I20260926 16:33:55.326815 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 22900
I20260926 16:33:55.332320 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 22513
I20260926 16:33:55.338428 11299 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260926 16:33:55.673012 11299 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260926 16:33:55.673183 11299 mini_ranger.cc:66] Stopping Ranger...
I20260926 16:33:56.017719 11299 mini_ranger.cc:68] Stopped Ranger
2026-09-26 16:33:56.017 UTC [22099] LOG: received smart shutdown request
2026-09-26 16:33:56.021 UTC [22099] LOG: background worker "logical replication launcher" (PID 22108) exited with exit code 1
2026-09-26 16:33:56.021 UTC [22103] LOG: shutting down
2026-09-26 16:33:56.185 UTC [22099] LOG: database system is shut down
2026-09-26T16:33:56Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (46007 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:33:56 dist-test-slave-2zvq krb5kdc[23091](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:33:56 dist-test-slave-2zvq krb5kdc[23091](info): set up 2 sockets
Sep 26 16:33:56 dist-test-slave-2zvq krb5kdc[23091](info): commencing operation
krb5kdc: starting...
W20260926 16:33:57.340009 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:33:57 dist-test-slave-2zvq krb5kdc[23091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:33:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:33:57Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.11.8.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.11.8.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.11.8.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/rangeradmin_127.11.8.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.11.8.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/rangeradmin_127.11.8.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.11.8.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.11.8.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.11.8.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/rangerlookup_127.11.8.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.11.8.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/rangerlookup_127.11.8.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.8.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.11.8.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.8.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/HTTP_127.11.8.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.8.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/HTTP_127.11.8.212@KRBTEST.COM.keytab.
I20260926 16:33:57.497530 11299 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskTzWGcB/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/postgres -l logfile start
2026-09-26 16:33:59.964 UTC [23121] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-26 16:33:59.964 UTC [23121] LOG: listening on IPv4 address "127.11.8.212", port 58371
2026-09-26 16:33:59.968 UTC [23121] LOG: listening on Unix socket "/tmp/.s.PGSQL.58371"
2026-09-26 16:33:59.988 UTC [23124] LOG: database system was shut down at 2026-09-26 16:33:58 UTC
2026-09-26 16:33:59.992 UTC [23121] LOG: database system is ready to accept connections
I20260926 16:34:00.947934 11299 mini_postgres.cc:96] Postgres bound to 58371
2026-09-26 16:34:00.954 UTC [23132] FATAL: database "slave" does not exist
127.11.8.212:58371 - accepting connections
I20260926 16:34:00.955278 11299 mini_ranger.cc:161] Starting Ranger...
I20260926 16:34:00.970046 11299 mini_ranger.cc:84] Created miniranger Postgres user
I20260926 16:34:01.432642 11299 mini_ranger.cc:87] Created ranger Postgres database
I20260926 16:34:01.432730 11299 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ranger-admin
2026-09-26 16:34:01,697 [I] DB FLAVOR :POSTGRES
2026-09-26 16:34:01,698 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 16:34:01,698 [I] Checking connection..
2026-09-26 16:34:01,698 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:58371/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-26 16:34:01,977 [I] Checking connection passed.
2026-09-26 16:34:01,977 [I] --------- Verifying version history table ---------
2026-09-26 16:34:01,977 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:58371/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 16:34:02,240 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 16:34:02,241 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:58371/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 16:34:02,500 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 16:34:02,501 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-26 16:34:02,501 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:58371/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-26 16:34:02.725 UTC [23196] WARNING: there is no transaction in progress
2026-09-26 16:34:02,731 [I] create_dbversion_catalog.sql file imported successfully
2026-09-26 16:34:02,731 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:58371/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 16:34:02,961 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-26 16:34:02,961 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 16:34:02,961 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:58371/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-26 16:34:03,217 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:58371/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-26 16:34:03,468 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:58371/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-2zvq.c.gcp-upstream.internal','N') ;"
2026-09-26 16:34:03,688 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:58371/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 16:34:03,952 [I] Table x_portal_user does not exist in database ranger
2026-09-26 16:34:03,952 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:58371/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 16:34:04,193 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-26 16:34:04,193 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:58371/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 16:34:04,424 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:58371/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 16:34:04,648 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-26 16:34:04,648 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:58371/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-26 16:34:05.599 UTC [23304] WARNING: there is no transaction in progress
2026-09-26 16:34:05.616 UTC [23304] WARNING: there is no transaction in progress
2026-09-26 16:34:05.632 UTC [23304] WARNING: there is no transaction in progress
2026-09-26 16:34:05.648 UTC [23304] WARNING: there is no transaction in progress
2026-09-26 16:34:05.665 UTC [23304] WARNING: there is no transaction in progress
2026-09-26 16:34:05.685 UTC [23304] WARNING: there is no transaction in progress
2026-09-26 16:34:05.745 UTC [23304] WARNING: there is no transaction in progress
2026-09-26 16:34:05.752 UTC [23304] WARNING: there is no transaction in progress
2026-09-26 16:34:05.766 UTC [23304] WARNING: there is no transaction in progress
2026-09-26 16:34:05.776 UTC [23304] WARNING: there is no transaction in progress
2026-09-26 16:34:05.785 UTC [23304] WARNING: there is no transaction in progress
2026-09-26 16:34:05.799 UTC [23304] WARNING: there is no transaction in progress
2026-09-26 16:34:05.809 UTC [23304] WARNING: there is no transaction in progress
2026-09-26 16:34:05.815 UTC [23304] WARNING: there is no transaction in progress
2026-09-26 16:34:05.822 UTC [23304] WARNING: there is no transaction in progress
2026-09-26 16:34:05.828 UTC [23304] WARNING: there is no transaction in progress
2026-09-26 16:34:06,440 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-26 16:34:06,440 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:58371/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-26 16:34:06,687 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-26 16:34:06,687 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:58371/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 16:34:06,944 [I] Table x_portal_user already exists in database 'ranger'
2026-09-26 16:34:06,945 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:58371/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 16:34:07,173 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-26 16:34:07,174 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:58371/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 16:34:07,402 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:58371/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 16:34:07,637 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:58371/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-2zvq.c.gcp-upstream.internal';"
2026-09-26 16:34:07,875 [I] CORE_DB_SCHEMA import status has been updated
2026-09-26 16:34:07,875 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:58371/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-26 16:34:08,135 [I] DB_PATCHES have already been applied
I20260926 16:34:08.141876 11299 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ranger-admin:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskTzWGcB/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260926 16:34:08.141942 11299 mini_ranger.cc:193] Using host: 127.11.8.212
I20260926 16:34:08.145426 11299 mini_ranger.cc:227] Ranger admin URL: http://127.11.8.212:56655
Sep 26, 2026 4:34:08 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 4:34:08 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 4:34:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ranger-admin/ews/webapp
Sep 26, 2026 4:34:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 26, 2026 4:34:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 26, 2026 4:34:09 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ranger-admin/ews/webapp].
Sep 26, 2026 4:34:09 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260926 16:34:09.249509 11299 mini_ranger.cc:160] Time spent starting Ranger: real 8.294s user 0.000s sys 0.009s
Sep 26, 2026 4:34:28 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 26, 2026 4:34:28 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Sep 26, 2026 4:34:28 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 26, 2026 4:34:28 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 26, 2026 4:34:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 26, 2026 4:34:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 26, 2026 4:34:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 26, 2026 4:34:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 26, 2026 4:34:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 26, 2026 4:34:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 26, 2026 4:34:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 26, 2026 4:34:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 26, 2026 4:34:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 26, 2026 4:34:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 26, 2026 4:34:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 26, 2026 4:34:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 26, 2026 4:34:28 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 26, 2026 4:34:29 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260926 16:34:29.592324 11299 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.11.8.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.11.8.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.11.8.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/rangerkms_127.11.8.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.11.8.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/rangerkms_127.11.8.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.8.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.11.8.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.8.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/HTTP_127.11.8.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.8.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/HTTP_127.11.8.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 26 16:34:29 dist-test-slave-2zvq krb5kdc[23091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440469, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260926 16:34:29.685877 11299 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260926 16:34:29.695503 11299 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260926 16:34:30.542539 11299 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260926 16:34:30.542629 11299 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ranger-kms
I20260926 16:34:30.542651 11299 mini_ranger_kms.cc:227] Using postgres at 127.11.8.212:58371
2026-09-26 16:34:30,711 [I] DB FLAVOR :POSTGRES
2026-09-26 16:34:30,711 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 16:34:30,711 [I] Checking connection
2026-09-26 16:34:30,712 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:58371/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-26 16:34:30,968 [I] connection success
2026-09-26 16:34:30,968 [I] --------- Verifying Ranger DB tables ---------
2026-09-26 16:34:30,968 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-26 16:34:30,968 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:58371/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-26 16:34:31,277 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-26 16:34:31,278 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 16:34:31,278 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-26 16:34:31,278 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.212:58371/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-26 16:34:31,578 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260926 16:34:32.070765 11299 mini_ranger_kms.cc:310] Created kms service
I20260926 16:34:32.296157 11299 mini_ranger_kms.cc:326] Created kudu user
I20260926 16:34:32.338928 11299 mini_ranger_kms.cc:343] Created rangerkms user
Sep 26 16:34:32 dist-test-slave-2zvq krb5kdc[23091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440469, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.11.8.212@KRBTEST.COM
I20260926 16:34:32.721490 11299 mini_ranger_kms.cc:384] Added ranger policy
I20260926 16:34:32.721626 11299 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ranger-kms:/tmp/dist-test-taskTzWGcB/build/debug/bin/postgresql.jar:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskTzWGcB/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260926 16:34:32.721647 11299 mini_ranger_kms.cc:240] Using host: 127.11.8.212
I20260926 16:34:32.725572 11299 mini_ranger_kms.cc:276] Ranger KMS PID: 23484
I20260926 16:34:32.725653 11299 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.11.8.212:54903
Sep 26, 2026 4:34:33 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 4:34:33 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 4:34:33 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 26, 2026 4:34:33 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 26, 2026 4:34:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ranger-kms/ews/webapp].
Sep 26, 2026 4:34:34 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Sep 26 16:34:34 dist-test-slave-2zvq krb5kdc[23091](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790440474, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.8.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:34:35.014220 11299 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.328s user 0.001s sys 0.152s
I20260926 16:34:35.014361 11299 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260926 16:34:35.014424 11299 mini_ranger_kms.cc:401] 127.11.8.212:54903/kms/v1/keys
Sep 26 16:34:35 dist-test-slave-2zvq krb5kdc[23091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440474, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.8.212@KRBTEST.COM for HTTP/127.11.8.212@KRBTEST.COM
Sep 26 16:34:35 dist-test-slave-2zvq krb5kdc[23091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440474, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.8.212@KRBTEST.COM for HTTP/127.11.8.212@KRBTEST.COM
Sep 26 16:34:37 dist-test-slave-2zvq krb5kdc[23091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440477, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.11.8.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
I20260926 16:34:37.993607 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:37397
--webserver_interface=127.11.8.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:34155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:37397
--ranger_config_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.8.212:54903/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:34:38.107117 23535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:34:38.107403 23535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:34:38.107483 23535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:34:38.111608 23535 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:34:38.111680 23535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:34:38.111734 23535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:34:38.111786 23535 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:34:38.111824 23535 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:34:38.117239 23535 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:34155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/master-0/wal
--ranger_kms_url=127.11.8.212:54903/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:37397
--ranger_config_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:37397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:34:38.118362 23535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:34:38.119496 23535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:34:38.129014 23543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:34:38.129101 23540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:34:38.129010 23541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:34:38.129701 23535 server_base.cc:1034] running on GCE node
I20260926 16:34:38.130340 23535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:34:38.131487 23535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:34:38.132665 23535 hybrid_clock.cc:648] HybridClock initialized: now 1790440478132642 us; error 42 us; skew 500 ppm
Sep 26 16:34:38 dist-test-slave-2zvq krb5kdc[23091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440478, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:34:38.140095 23535 init.cc:377] Logged in from keytab as kudu/127.11.8.254@KRBTEST.COM (short username kudu)
I20260926 16:34:38.141271 23535 webserver.cc:460] Webserver started at http://127.11.8.254:43123/ using document root <none> and password file <none>
I20260926 16:34:38.141953 23535 fs_manager.cc:359] Metadata directory not provided
I20260926 16:34:38.142031 23535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:34:38.142257 23535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 16:34:38 dist-test-slave-2zvq krb5kdc[23091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440478, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for HTTP/127.11.8.212@KRBTEST.COM
I20260926 16:34:38.187433 23535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "309a8bd2d36144d78a6db0864f78409e"
format_stamp: "Formatted at 2026-09-26 16:34:38 on dist-test-slave-2zvq"
server_key: "72b14cf06454241b420e7e91e83a8e3e"
server_key_iv: "455954f022a8d2c3bf7424779ef55622"
server_key_version: "kuduclusterkey@0"
I20260926 16:34:38.188038 23535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "309a8bd2d36144d78a6db0864f78409e"
format_stamp: "Formatted at 2026-09-26 16:34:38 on dist-test-slave-2zvq"
server_key: "72b14cf06454241b420e7e91e83a8e3e"
server_key_iv: "455954f022a8d2c3bf7424779ef55622"
server_key_version: "kuduclusterkey@0"
I20260926 16:34:38.192148 23535 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260926 16:34:38.209342 23550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:38.210831 23535 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:34:38.210966 23535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "309a8bd2d36144d78a6db0864f78409e"
format_stamp: "Formatted at 2026-09-26 16:34:38 on dist-test-slave-2zvq"
server_key: "72b14cf06454241b420e7e91e83a8e3e"
server_key_iv: "455954f022a8d2c3bf7424779ef55622"
server_key_version: "kuduclusterkey@0"
I20260926 16:34:38.211097 23535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:34:38.230439 23535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:34:38.233449 23535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:34:38.233666 23535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:34:38.241897 23535 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:37397
I20260926 16:34:38.241904 23602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:37397 every 8 connection(s)
I20260926 16:34:38.242924 23535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:34:38.245622 23603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:34:38.251133 23603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e: Bootstrap starting.
I20260926 16:34:38.251166 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 23535
I20260926 16:34:38.251281 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:34:38.251593 11299 external_mini_cluster.cc:1428] Setting key 589b66da4e7e0e31682454bbc210a414
I20260926 16:34:38.253834 23603 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e: Neither blocks nor log segments found. Creating new log.
I20260926 16:34:38.254644 23603 log.cc:824] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e: Log is configured to *not* fsync() on all Append() calls
I20260926 16:34:38.257114 23603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e: No bootstrap required, opened a new log
I20260926 16:34:38.261554 23603 raft_consensus.cc:348] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "309a8bd2d36144d78a6db0864f78409e" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37397 } }
I20260926 16:34:38.261798 23603 raft_consensus.cc:374] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:34:38.261878 23603 raft_consensus.cc:729] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 309a8bd2d36144d78a6db0864f78409e, State: Initialized, Role: FOLLOWER
Sep 26 16:34:38 dist-test-slave-2zvq krb5kdc[23091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440477, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:34:38.262426 23603 consensus_queue.cc:260] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "309a8bd2d36144d78a6db0864f78409e" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37397 } }
I20260926 16:34:38.262578 23603 raft_consensus.cc:388] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:34:38.262670 23603 raft_consensus.cc:482] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:34:38.262784 23603 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:34:38.263995 23603 raft_consensus.cc:504] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "309a8bd2d36144d78a6db0864f78409e" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37397 } }
I20260926 16:34:38.264386 23603 leader_election.cc:302] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 309a8bd2d36144d78a6db0864f78409e; no voters:
I20260926 16:34:38.264731 23603 leader_election.cc:288] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:34:38.264822 23608 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:34:38.265131 23608 raft_consensus.cc:686] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e [term 1 LEADER]: Becoming Leader. State: Replica: 309a8bd2d36144d78a6db0864f78409e, State: Running, Role: LEADER
I20260926 16:34:38.265489 23608 consensus_queue.cc:237] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "309a8bd2d36144d78a6db0864f78409e" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37397 } }
I20260926 16:34:38.266211 23603 sys_catalog.cc:564] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:34:38.266804 23608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "309a8bd2d36144d78a6db0864f78409e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "309a8bd2d36144d78a6db0864f78409e" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37397 } } }
I20260926 16:34:38.266994 23608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e [sys.catalog]: This master's current role is: LEADER
I20260926 16:34:38.267537 23608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 309a8bd2d36144d78a6db0864f78409e. Latest consensus state: current_term: 1 leader_uuid: "309a8bd2d36144d78a6db0864f78409e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "309a8bd2d36144d78a6db0864f78409e" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37397 } } }
I20260926 16:34:38.267711 23608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e [sys.catalog]: This master's current role is: LEADER
I20260926 16:34:38.268142 23615 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:34:38.269055 23606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:38.252995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60118 (local address 127.11.8.254:37397)
0926 16:34:38.253511 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:38.253530 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:38.253575 (+ 45us) server_negotiation.cc:407] Connection header received
0926 16:34:38.254440 (+ 865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:38.254466 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:38.254901 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:38.255278 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:34:38.256027 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:38.257572 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:38.258859 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:38.259570 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:38.259741 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:38.262884 (+ 3143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:34:38.262891 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:34:38.262912 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:34:38.262943 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:34:38.265131 (+ 2188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:38.265655 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:38.265660 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:38.265665 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:38.265789 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:38.266111 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:38.266114 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:38.266116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:38.266656 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:34:38.267535 (+ 879us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:38.267711 (+ 176us) server_negotiation.cc:299] Negotiation successful
0926 16:34:38.268016 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":126,"threads_started":1}
I20260926 16:34:38.271538 23615 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:34:38.272293 23603 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260926 16:34:38.284090 23615 catalog_manager.cc:1347] Generated new cluster ID: ace32a5f1d0347aabf07713529130ff8
I20260926 16:34:38.284163 23615 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:34:38.300100 23615 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:34:38.301503 23615 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:34:38.312410 23615 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e: Generated new TSK 0
I20260926 16:34:38.313294 23615 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.8.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab.
Entry for principal kudu/127.11.8.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab.
I20260926 16:34:38.992985 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.8.212:54903/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.193:0
--local_ip_for_outbound_sockets=127.11.8.193
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:37397
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:34155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:34:39.117333 23648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:34:39.117612 23648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:34:39.117744 23648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:34:39.122262 23648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:34:39.122351 23648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:34:39.122434 23648 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.193
I20260926 16:34:39.128225 23648 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:34155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.11.8.212:54903/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:37397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23648
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.193
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:34:39.129285 23648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:34:39.130451 23648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:34:39.140034 23654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:34:39.140049 23656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:34:39.140184 23653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:34:39.140698 23648 server_base.cc:1034] running on GCE node
I20260926 16:34:39.141098 23648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:34:39.141698 23648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:34:39.142884 23648 hybrid_clock.cc:648] HybridClock initialized: now 1790440479142860 us; error 41 us; skew 500 ppm
Sep 26 16:34:39 dist-test-slave-2zvq krb5kdc[23091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440479, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:34:39.147614 23648 init.cc:377] Logged in from keytab as kudu/127.11.8.193@KRBTEST.COM (short username kudu)
I20260926 16:34:39.149047 23648 webserver.cc:460] Webserver started at http://127.11.8.193:41897/ using document root <none> and password file <none>
I20260926 16:34:39.149731 23648 fs_manager.cc:359] Metadata directory not provided
I20260926 16:34:39.149865 23648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:34:39.150117 23648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 16:34:39 dist-test-slave-2zvq krb5kdc[23091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440479, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for HTTP/127.11.8.212@KRBTEST.COM
I20260926 16:34:39.163892 23648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-0/data/instance:
uuid: "349e20badcc1476b905b8c4cc0119e24"
format_stamp: "Formatted at 2026-09-26 16:34:39 on dist-test-slave-2zvq"
server_key: "435a2a4b53d24b86e9d097e9a475e03d"
server_key_iv: "be4cb74487516cdad84adbb34ff75d51"
server_key_version: "kuduclusterkey@0"
I20260926 16:34:39.164558 23648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance:
uuid: "349e20badcc1476b905b8c4cc0119e24"
format_stamp: "Formatted at 2026-09-26 16:34:39 on dist-test-slave-2zvq"
server_key: "435a2a4b53d24b86e9d097e9a475e03d"
server_key_iv: "be4cb74487516cdad84adbb34ff75d51"
server_key_version: "kuduclusterkey@0"
I20260926 16:34:39.169660 23648 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 16:34:39.186564 23665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:39.188220 23648 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 16:34:39.188346 23648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-0/wal
uuid: "349e20badcc1476b905b8c4cc0119e24"
format_stamp: "Formatted at 2026-09-26 16:34:39 on dist-test-slave-2zvq"
server_key: "435a2a4b53d24b86e9d097e9a475e03d"
server_key_iv: "be4cb74487516cdad84adbb34ff75d51"
server_key_version: "kuduclusterkey@0"
I20260926 16:34:39.188472 23648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:34:39.204509 23648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:34:39.208233 23648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:34:39.208422 23648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:34:39.209096 23648 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:34:39.210155 23648 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:34:39.210238 23648 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:39.210386 23648 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:34:39.210424 23648 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:39.221299 23648 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.193:33721
I20260926 16:34:39.222711 23648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
I20260926 16:34:39.225131 23778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.193:33721 every 8 connection(s)
I20260926 16:34:39.232467 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 23648
I20260926 16:34:39.232595 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance
I20260926 16:34:39.232908 11299 external_mini_cluster.cc:1428] Setting key 6970006179f861acc3fabdc38e5fca17
Sep 26 16:34:39 dist-test-slave-2zvq krb5kdc[23091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440479, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:34:39.267143 23781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:39.228018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37397 (local address 127.11.8.193:59667)
0926 16:34:39.234775 (+ 6757us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:39.234835 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0926 16:34:39.235857 (+ 1022us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:34:39.245679 (+ 9822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:34:39.245689 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:34:39.246045 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:34:39.246754 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:39.246774 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:39.247990 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:39.247994 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:39.249367 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:39.249379 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:39.250099 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:39.250102 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:39.250326 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:39.251015 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:34:39.251035 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:34:39.252789 (+ 1754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:34:39.263611 (+ 10822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:39.263618 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:39.263635 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:39.264076 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:39.264456 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:39.264460 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:39.264462 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:39.264671 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:39.265148 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:34:39.265156 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:34:39.265479 (+ 323us) client_negotiation.cc:769] Sending connection context
0926 16:34:39.265677 (+ 198us) client_negotiation.cc:240] Negotiation successful
0926 16:34:39.266022 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6517,"thread_start_us":150,"threads_started":1}
I20260926 16:34:39.267952 23782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:39.229141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:59667 (local address 127.11.8.254:37397)
0926 16:34:39.245213 (+ 16072us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:39.245221 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:39.245237 (+ 16us) server_negotiation.cc:407] Connection header received
0926 16:34:39.245279 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:39.245282 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:39.245330 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:39.246901 (+ 1571us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:34:39.246998 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:39.247842 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:39.249513 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:39.249851 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:39.249935 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:39.260941 (+ 11006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:34:39.260945 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:34:39.260947 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:34:39.260975 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:34:39.263431 (+ 2456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:39.264252 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:39.264256 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:39.264258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:39.264349 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:39.264791 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:39.264794 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:39.264796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:39.265026 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:34:39.267569 (+ 2543us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:39.267655 (+ 86us) server_negotiation.cc:299] Negotiation successful
0926 16:34:39.267790 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15970,"thread_start_us":81,"threads_started":1}
I20260926 16:34:39.269243 23779 heartbeater.cc:347] Connected to a master server at 127.11.8.254:37397
I20260926 16:34:39.269498 23779 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.11.8.194@KRBTEST.COM; defaulting to no policy
I20260926 16:34:39.270143 23779 heartbeater.cc:511] Master 127.11.8.254:37397 requested a full tablet report, sending...
I20260926 16:34:39.271929 23567 ts_manager.cc:194] Registered new tserver with Master: 349e20badcc1476b905b8c4cc0119e24 (127.11.8.193:33721)
I20260926 16:34:39.273512 23567 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.193@KRBTEST.COM'} at 127.11.8.193:59667
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab.
Entry for principal kudu/127.11.8.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab.
I20260926 16:34:39.299804 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.8.212:54903/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.194:0
--local_ip_for_outbound_sockets=127.11.8.194
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:37397
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:34155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 16:34:39 dist-test-slave-2zvq krb5kdc[23091](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790440479, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260926 16:34:39.447690 23787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:34:39.448042 23787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:34:39.448133 23787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:34:39.454320 23787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:34:39.454389 23787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:34:39.454478 23787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.194
I20260926 16:34:39.462253 23787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:34155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.11.8.212:54903/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:37397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.194
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:34:39.463598 23787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:34:39.464905 23787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:34:39.476569 23793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:34:39.476730 23787 server_base.cc:1034] running on GCE node
W20260926 16:34:39.476635 23796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:34:39.476568 23794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:34:39.477440 23787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:34:39.478180 23787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:34:39.479522 23787 hybrid_clock.cc:648] HybridClock initialized: now 1790440479479503 us; error 34 us; skew 500 ppm
Sep 26 16:34:39 dist-test-slave-2zvq krb5kdc[23091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440479, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:34:39.482402 23787 init.cc:377] Logged in from keytab as kudu/127.11.8.194@KRBTEST.COM (short username kudu)
I20260926 16:34:39.483345 23787 webserver.cc:460] Webserver started at http://127.11.8.194:43625/ using document root <none> and password file <none>
I20260926 16:34:39.483889 23787 fs_manager.cc:359] Metadata directory not provided
I20260926 16:34:39.483975 23787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:34:39.484192 23787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 16:34:39 dist-test-slave-2zvq krb5kdc[23091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440479, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for HTTP/127.11.8.212@KRBTEST.COM
I20260926 16:34:39.497574 23787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-1/data/instance:
uuid: "225c06bebce04085b3d1fe0b93d46725"
format_stamp: "Formatted at 2026-09-26 16:34:39 on dist-test-slave-2zvq"
server_key: "134b1499e895b9ccb12c1423862d1da5"
server_key_iv: "053e5c3284cd8fe6c911e40d0fe74d19"
server_key_version: "kuduclusterkey@0"
I20260926 16:34:39.498848 23787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance:
uuid: "225c06bebce04085b3d1fe0b93d46725"
format_stamp: "Formatted at 2026-09-26 16:34:39 on dist-test-slave-2zvq"
server_key: "134b1499e895b9ccb12c1423862d1da5"
server_key_iv: "053e5c3284cd8fe6c911e40d0fe74d19"
server_key_version: "kuduclusterkey@0"
I20260926 16:34:39.507745 23787 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.004s sys 0.000s
I20260926 16:34:39.534922 23803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:39.536729 23787 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260926 16:34:39.536875 23787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-1/wal
uuid: "225c06bebce04085b3d1fe0b93d46725"
format_stamp: "Formatted at 2026-09-26 16:34:39 on dist-test-slave-2zvq"
server_key: "134b1499e895b9ccb12c1423862d1da5"
server_key_iv: "053e5c3284cd8fe6c911e40d0fe74d19"
server_key_version: "kuduclusterkey@0"
I20260926 16:34:39.537040 23787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:34:39.554086 23787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:34:39.557583 23787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:34:39.557773 23787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:34:39.558480 23787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:34:39.559520 23787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:34:39.559585 23787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:39.559656 23787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:34:39.559688 23787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:39.570025 23787 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.194:43557
I20260926 16:34:39.571213 23787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
I20260926 16:34:39.579041 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 23787
I20260926 16:34:39.579166 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance
I20260926 16:34:39.579471 11299 external_mini_cluster.cc:1428] Setting key 39613eb3c2bf93e69b063e09ac07378f
I20260926 16:34:39.580053 23916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.194:43557 every 8 connection(s)
Sep 26 16:34:39 dist-test-slave-2zvq krb5kdc[23091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440479, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.8.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.195@KRBTEST.COM" created.
I20260926 16:34:39.607190 23782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:39.584749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:53143 (local address 127.11.8.254:37397)
0926 16:34:39.584898 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:39.584903 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:39.585264 (+ 361us) server_negotiation.cc:407] Connection header received
0926 16:34:39.586531 (+ 1267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:39.586535 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:39.586594 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:39.586650 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:34:39.588299 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:39.589143 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:39.590850 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:39.591226 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:39.591317 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:39.600806 (+ 9489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:34:39.600810 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:34:39.600813 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:34:39.600851 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:34:39.603080 (+ 2229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:39.605099 (+ 2019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:39.605103 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:39.605105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:39.605199 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:39.605682 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:39.605685 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:39.605687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:39.605933 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:34:39.606027 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:39.606869 (+ 842us) server_negotiation.cc:299] Negotiation successful
0926 16:34:39.607012 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 16:34:39.608448 23920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:39.583621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37397 (local address 127.11.8.194:53143)
0926 16:34:39.585112 (+ 1491us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:39.585169 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 16:34:39.586248 (+ 1079us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:34:39.587087 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:34:39.587096 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:34:39.587454 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:34:39.588129 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:39.588148 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:39.589298 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:39.589301 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:39.590694 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:39.590709 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:39.591523 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:39.591527 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:39.591731 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:39.592738 (+ 1007us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:34:39.592759 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:34:39.600532 (+ 7773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:34:39.604466 (+ 3934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:39.604474 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:39.604496 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:39.604951 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:39.605329 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:39.605332 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:39.605334 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:39.605556 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:39.606111 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:34:39.606118 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:34:39.606522 (+ 404us) client_negotiation.cc:769] Sending connection context
0926 16:34:39.606773 (+ 251us) client_negotiation.cc:240] Negotiation successful
0926 16:34:39.607189 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1251,"thread_start_us":172,"threads_started":1}
I20260926 16:34:39.610092 23917 heartbeater.cc:347] Connected to a master server at 127.11.8.254:37397
I20260926 16:34:39.610383 23917 heartbeater.cc:464] Registering TS with master...
I20260926 16:34:39.611006 23917 heartbeater.cc:511] Master 127.11.8.254:37397 requested a full tablet report, sending...
I20260926 16:34:39.612166 23567 ts_manager.cc:194] Registered new tserver with Master: 225c06bebce04085b3d1fe0b93d46725 (127.11.8.194:43557)
I20260926 16:34:39.612905 23567 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.194@KRBTEST.COM'} at 127.11.8.194:53143
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab.
Entry for principal kudu/127.11.8.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab.
I20260926 16:34:39.630443 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.8.212:54903/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.195:0
--local_ip_for_outbound_sockets=127.11.8.195
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:37397
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:34155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:34:39.791937 23922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:34:39.792356 23922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:34:39.792469 23922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:34:39.798296 23922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:34:39.798441 23922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:34:39.798568 23922 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.195
I20260926 16:34:39.806476 23922 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:34155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.11.8.212:54903/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:37397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.195
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:34:39.808019 23922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:34:39.809366 23922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:34:39.822355 23928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:34:39.822355 23930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:34:39.822957 23927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:34:39.823501 23922 server_base.cc:1034] running on GCE node
I20260926 16:34:39.824110 23922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:34:39.824884 23922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:34:39.826076 23922 hybrid_clock.cc:648] HybridClock initialized: now 1790440479826009 us; error 79 us; skew 500 ppm
Sep 26 16:34:39 dist-test-slave-2zvq krb5kdc[23091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440479, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:34:39.829727 23922 init.cc:377] Logged in from keytab as kudu/127.11.8.195@KRBTEST.COM (short username kudu)
I20260926 16:34:39.830792 23922 webserver.cc:460] Webserver started at http://127.11.8.195:45987/ using document root <none> and password file <none>
I20260926 16:34:39.831463 23922 fs_manager.cc:359] Metadata directory not provided
I20260926 16:34:39.831533 23922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:34:39.831763 23922 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 16:34:39 dist-test-slave-2zvq krb5kdc[23091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440479, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for HTTP/127.11.8.212@KRBTEST.COM
I20260926 16:34:39.848702 23922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-2/data/instance:
uuid: "2f835f93a0e04a30bdbd13c1586c109d"
format_stamp: "Formatted at 2026-09-26 16:34:39 on dist-test-slave-2zvq"
server_key: "49a666427b8d88b0827b46bf2557db67"
server_key_iv: "61655cfbe4e6ae405824383b8614e8d8"
server_key_version: "kuduclusterkey@0"
I20260926 16:34:39.850523 23922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance:
uuid: "2f835f93a0e04a30bdbd13c1586c109d"
format_stamp: "Formatted at 2026-09-26 16:34:39 on dist-test-slave-2zvq"
server_key: "49a666427b8d88b0827b46bf2557db67"
server_key_iv: "61655cfbe4e6ae405824383b8614e8d8"
server_key_version: "kuduclusterkey@0"
I20260926 16:34:39.855880 23922 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
Sep 26 16:34:39 dist-test-slave-2zvq krb5kdc[23091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440479, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for HTTP/127.11.8.212@KRBTEST.COM
I20260926 16:34:39.890702 23937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:39.892596 23922 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.002s sys 0.000s
I20260926 16:34:39.892756 23922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-2/wal
uuid: "2f835f93a0e04a30bdbd13c1586c109d"
format_stamp: "Formatted at 2026-09-26 16:34:39 on dist-test-slave-2zvq"
server_key: "49a666427b8d88b0827b46bf2557db67"
server_key_iv: "61655cfbe4e6ae405824383b8614e8d8"
server_key_version: "kuduclusterkey@0"
I20260926 16:34:39.892915 23922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:34:39.923657 23922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:34:39.927623 23922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:34:39.927872 23922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:34:39.928591 23922 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:34:39.929629 23922 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:34:39.929744 23922 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:39.929839 23922 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:34:39.929901 23922 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:39.940083 23922 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.195:34329
I20260926 16:34:39.941398 23922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
I20260926 16:34:39.946650 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 23922
I20260926 16:34:39.946780 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance
I20260926 16:34:39.947057 11299 external_mini_cluster.cc:1428] Setting key 638c4c6851a7a29aa8516c950f7df14d
I20260926 16:34:39.953150 24050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.195:34329 every 8 connection(s)
Sep 26 16:34:39 dist-test-slave-2zvq krb5kdc[23091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440479, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:34:39.993525 23782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:39.957301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:50777 (local address 127.11.8.254:37397)
0926 16:34:39.957420 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:39.957425 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:39.967696 (+ 10271us) server_negotiation.cc:407] Connection header received
0926 16:34:39.969017 (+ 1321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:39.969021 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:39.969074 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:39.969125 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:34:39.970845 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:39.971647 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:39.973301 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:39.973638 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:39.973706 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:39.988507 (+ 14801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:34:39.988511 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:34:39.988514 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:34:39.988543 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:34:39.990208 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:39.990877 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:39.990880 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:39.990882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:39.990960 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:39.991353 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:39.991356 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:39.991357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:39.991568 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:34:39.991654 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:39.993245 (+ 1591us) server_negotiation.cc:299] Negotiation successful
0926 16:34:39.993368 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 16:34:39.994884 24053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:39.956133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37397 (local address 127.11.8.195:50777)
0926 16:34:39.967493 (+ 11360us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:39.967577 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0926 16:34:39.968752 (+ 1175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:34:39.969578 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:34:39.969588 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:34:39.969964 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:34:39.970684 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:39.970702 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:39.971804 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:39.971807 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:39.973157 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:39.973168 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:39.973897 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:39.973900 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:39.974092 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:39.986469 (+ 12377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:34:39.986495 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:34:39.988301 (+ 1806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:34:39.990394 (+ 2093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:39.990401 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:39.990421 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:39.990765 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:39.991051 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:39.991054 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:39.991056 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:39.991250 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:39.991708 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:34:39.991715 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:34:39.992068 (+ 353us) client_negotiation.cc:769] Sending connection context
0926 16:34:39.992298 (+ 230us) client_negotiation.cc:240] Negotiation successful
0926 16:34:39.992662 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11044,"thread_start_us":170,"threads_started":1}
I20260926 16:34:39.996495 24051 heartbeater.cc:347] Connected to a master server at 127.11.8.254:37397
I20260926 16:34:39.996740 24051 heartbeater.cc:464] Registering TS with master...
I20260926 16:34:39.997308 24051 heartbeater.cc:511] Master 127.11.8.254:37397 requested a full tablet report, sending...
I20260926 16:34:39.998528 23566 ts_manager.cc:194] Registered new tserver with Master: 2f835f93a0e04a30bdbd13c1586c109d (127.11.8.195:34329)
I20260926 16:34:39.999353 23566 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.195@KRBTEST.COM'} at 127.11.8.195:50777
I20260926 16:34:40.007392 11299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 16:34:40.008249 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 23648
I20260926 16:34:40.017107 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 23787
I20260926 16:34:40.025173 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 23922
I20260926 16:34:40.031517 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 23535
I20260926 16:34:40.038839 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:37397
--webserver_interface=127.11.8.254
--webserver_port=43123
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:34155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:37397
--ranger_config_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.8.212:54903/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.8.212:54903/kms with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:34:40.154055 24054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:34:40.154358 24054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:34:40.154451 24054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:34:40.158471 24054 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:34:40.158519 24054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:34:40.158537 24054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:34:40.158553 24054 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:34:40.158592 24054 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:34:40.163830 24054 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:34155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.8.212:54903/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:37397
--ranger_config_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:37397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=43123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:34:40.164789 24054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:34:40.165833 24054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:34:40.174178 24062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:34:40.174243 24054 server_base.cc:1034] running on GCE node
W20260926 16:34:40.174260 24059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:34:40.174175 24060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:34:40.175048 24054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:34:40.176111 24054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:34:40.177318 24054 hybrid_clock.cc:648] HybridClock initialized: now 1790440480177290 us; error 39 us; skew 500 ppm
Sep 26 16:34:40 dist-test-slave-2zvq krb5kdc[23091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440480, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:34:40.180259 24054 init.cc:377] Logged in from keytab as kudu/127.11.8.254@KRBTEST.COM (short username kudu)
I20260926 16:34:40.181088 24054 webserver.cc:460] Webserver started at http://127.11.8.254:43123/ using document root <none> and password file <none>
I20260926 16:34:40.181613 24054 fs_manager.cc:359] Metadata directory not provided
I20260926 16:34:40.181708 24054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 16:34:40 dist-test-slave-2zvq krb5kdc[23091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440480, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for HTTP/127.11.8.212@KRBTEST.COM
I20260926 16:34:40.245628 24054 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 16:34:40.248152 24070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:40.249461 24054 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260926 16:34:40.249607 24054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "309a8bd2d36144d78a6db0864f78409e"
format_stamp: "Formatted at 2026-09-26 16:34:38 on dist-test-slave-2zvq"
server_key: "72b14cf06454241b420e7e91e83a8e3e"
server_key_iv: "455954f022a8d2c3bf7424779ef55622"
server_key_version: "kuduclusterkey@0"
I20260926 16:34:40.250141 24054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:34:40.290091 24054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:34:40.293187 24054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:34:40.293695 24054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:34:40.300879 24054 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:37397
I20260926 16:34:40.300891 24122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:37397 every 8 connection(s)
I20260926 16:34:40.301942 24054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:34:40.306197 24123 sys_catalog.cc:263] Verifying existing consensus state
I20260926 16:34:40.307096 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 24054
I20260926 16:34:40.307525 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.8.212:54903/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.193:33721
--local_ip_for_outbound_sockets=127.11.8.193
--tserver_master_addrs=127.11.8.254:37397
--webserver_port=41897
--webserver_interface=127.11.8.193
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:34155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.8.212:54903/kms with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 16:34:40.307937 24123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e: Bootstrap starting.
I20260926 16:34:40.323839 24123 log.cc:824] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e: Log is configured to *not* fsync() on all Append() calls
I20260926 16:34:40.330251 24123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 16:34:40.330674 24123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e: Bootstrap complete.
I20260926 16:34:40.334825 24123 raft_consensus.cc:348] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "309a8bd2d36144d78a6db0864f78409e" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37397 } }
I20260926 16:34:40.335224 24123 raft_consensus.cc:729] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 309a8bd2d36144d78a6db0864f78409e, State: Initialized, Role: FOLLOWER
I20260926 16:34:40.335736 24123 consensus_queue.cc:260] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "309a8bd2d36144d78a6db0864f78409e" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37397 } }
I20260926 16:34:40.335878 24123 raft_consensus.cc:388] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:34:40.335966 24123 raft_consensus.cc:482] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:34:40.336297 24123 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e [term 1 FOLLOWER]: Advancing to term 2
I20260926 16:34:40.338219 24123 raft_consensus.cc:504] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "309a8bd2d36144d78a6db0864f78409e" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37397 } }
I20260926 16:34:40.338593 24123 leader_election.cc:302] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 309a8bd2d36144d78a6db0864f78409e; no voters:
I20260926 16:34:40.338915 24123 leader_election.cc:288] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 16:34:40.338994 24127 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e [term 2 FOLLOWER]: Leader election won for term 2
I20260926 16:34:40.339227 24127 raft_consensus.cc:686] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e [term 2 LEADER]: Becoming Leader. State: Replica: 309a8bd2d36144d78a6db0864f78409e, State: Running, Role: LEADER
I20260926 16:34:40.339504 24127 consensus_queue.cc:237] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "309a8bd2d36144d78a6db0864f78409e" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37397 } }
I20260926 16:34:40.340124 24123 sys_catalog.cc:564] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:34:40.340744 24128 sys_catalog.cc:455] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "309a8bd2d36144d78a6db0864f78409e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "309a8bd2d36144d78a6db0864f78409e" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37397 } } }
I20260926 16:34:40.340862 24128 sys_catalog.cc:458] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e [sys.catalog]: This master's current role is: LEADER
I20260926 16:34:40.341166 24129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 309a8bd2d36144d78a6db0864f78409e. Latest consensus state: current_term: 2 leader_uuid: "309a8bd2d36144d78a6db0864f78409e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "309a8bd2d36144d78a6db0864f78409e" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37397 } } }
I20260926 16:34:40.341256 24129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e [sys.catalog]: This master's current role is: LEADER
I20260926 16:34:40.341545 24133 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:34:40.344574 24133 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:34:40.345089 24123 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260926 16:34:40.345189 24133 catalog_manager.cc:1259] Loaded cluster ID: ace32a5f1d0347aabf07713529130ff8
I20260926 16:34:40.345255 24133 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:34:40.350736 24133 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:34:40.352134 24133 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e: Loaded TSK: 0
I20260926 16:34:40.353618 24133 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 16:34:40.444190 24125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:34:40.444518 24125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:34:40.444598 24125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:34:40.450451 24125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:34:40.450519 24125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:34:40.450604 24125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.193
I20260926 16:34:40.458143 24125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:34155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.8.212:54903/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.193:33721
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.193
--webserver_port=41897
--tserver_master_addrs=127.11.8.254:37397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.193
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:34:40.459447 24125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:34:40.460678 24125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:34:40.472138 24163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:34:40.472504 24160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:34:40.472590 24161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:34:40.473155 24125 server_base.cc:1034] running on GCE node
I20260926 16:34:40.473675 24125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:34:40.474421 24125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:34:40.477690 24125 hybrid_clock.cc:648] HybridClock initialized: now 1790440480477640 us; error 80 us; skew 500 ppm
Sep 26 16:34:40 dist-test-slave-2zvq krb5kdc[23091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440480, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:34:40.481302 24125 init.cc:377] Logged in from keytab as kudu/127.11.8.193@KRBTEST.COM (short username kudu)
I20260926 16:34:40.482402 24125 webserver.cc:460] Webserver started at http://127.11.8.193:41897/ using document root <none> and password file <none>
I20260926 16:34:40.483093 24125 fs_manager.cc:359] Metadata directory not provided
I20260926 16:34:40.483184 24125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 16:34:40 dist-test-slave-2zvq krb5kdc[23091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440480, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for HTTP/127.11.8.212@KRBTEST.COM
I20260926 16:34:40.506541 24125 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.004s sys 0.000s
I20260926 16:34:40.509768 24171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:40.511374 24125 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 16:34:40.511504 24125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-0/wal
uuid: "349e20badcc1476b905b8c4cc0119e24"
format_stamp: "Formatted at 2026-09-26 16:34:39 on dist-test-slave-2zvq"
server_key: "435a2a4b53d24b86e9d097e9a475e03d"
server_key_iv: "be4cb74487516cdad84adbb34ff75d51"
server_key_version: "kuduclusterkey@0"
I20260926 16:34:40.512204 24125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:34:40.531255 24125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:34:40.535547 24125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:34:40.535810 24125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:34:40.536540 24125 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:34:40.537724 24125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:34:40.537834 24125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:40.537963 24125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:34:40.538050 24125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:40.549012 24125 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.193:33721
I20260926 16:34:40.553731 24125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
I20260926 16:34:40.555965 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 24125
I20260926 16:34:40.556429 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.8.212:54903/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.194:43557
--local_ip_for_outbound_sockets=127.11.8.194
--tserver_master_addrs=127.11.8.254:37397
--webserver_port=43625
--webserver_interface=127.11.8.194
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:34155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.8.212:54903/kms with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 16:34:40.555503 24284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.193:33721 every 8 connection(s)
Sep 26 16:34:40 dist-test-slave-2zvq krb5kdc[23091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440480, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:34:40.593475 24289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:40.560760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37397 (local address 127.11.8.193:39915)
0926 16:34:40.565130 (+ 4370us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:40.565195 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0926 16:34:40.566528 (+ 1333us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:34:40.568769 (+ 2241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:34:40.568782 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:34:40.569209 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:34:40.570012 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:40.570039 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:40.572365 (+ 2326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:40.572371 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:40.574161 (+ 1790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:40.574177 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:40.575831 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:40.575837 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:40.576053 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:40.576810 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:34:40.576835 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:34:40.582085 (+ 5250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:34:40.589107 (+ 7022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:40.589115 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:40.589133 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:40.589572 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:40.590059 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:40.590063 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:40.590065 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:40.590303 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:40.591240 (+ 937us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:34:40.591248 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:34:40.591604 (+ 356us) client_negotiation.cc:769] Sending connection context
0926 16:34:40.591838 (+ 234us) client_negotiation.cc:240] Negotiation successful
0926 16:34:40.592239 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4117,"thread_start_us":146,"threads_started":1}
I20260926 16:34:40.595718 24288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:40.560542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:39915 (local address 127.11.8.254:37397)
0926 16:34:40.565742 (+ 5200us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:40.565768 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:40.565818 (+ 50us) server_negotiation.cc:407] Connection header received
0926 16:34:40.567223 (+ 1405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:40.567243 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:40.567740 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:40.568157 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:34:40.570252 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:40.572172 (+ 1920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:40.574344 (+ 2172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:40.575413 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:40.575600 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:40.582470 (+ 6870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:34:40.582479 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:34:40.582500 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:34:40.582545 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:34:40.588899 (+ 6354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:40.589745 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:40.589753 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:40.589759 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:40.589930 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:40.590433 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:40.590437 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:40.590439 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:40.591007 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:34:40.591148 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:40.594387 (+ 3239us) server_negotiation.cc:299] Negotiation successful
0926 16:34:40.594719 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5040,"thread_start_us":123,"threads_started":1}
I20260926 16:34:40.598322 24285 heartbeater.cc:347] Connected to a master server at 127.11.8.254:37397
I20260926 16:34:40.598516 24285 heartbeater.cc:464] Registering TS with master...
I20260926 16:34:40.599104 24285 heartbeater.cc:511] Master 127.11.8.254:37397 requested a full tablet report, sending...
W20260926 16:34:40.601420 24285 heartbeater.cc:644] Failed to heartbeat to 127.11.8.254:37397 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 16:34:40.601534 24285 heartbeater.cc:464] Registering TS with master...
I20260926 16:34:40.601706 24285 heartbeater.cc:511] Master 127.11.8.254:37397 requested a full tablet report, sending...
I20260926 16:34:40.602631 24285 heartbeater.cc:464] Registering TS with master...
I20260926 16:34:40.602792 24285 heartbeater.cc:511] Master 127.11.8.254:37397 requested a full tablet report, sending...
W20260926 16:34:40.603564 24285 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 16:34:40.702425 24287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:34:40.702679 24287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:34:40.702747 24287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:34:40.706964 24287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:34:40.707038 24287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:34:40.707146 24287 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.194
I20260926 16:34:40.712643 24287 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:34155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.8.212:54903/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.194:43557
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.194
--webserver_port=43625
--tserver_master_addrs=127.11.8.254:37397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.194
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:34:40.713764 24287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:34:40.714844 24287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:34:40.725356 24294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:34:40.725461 24295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:34:40.725386 24297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:34:40.725942 24287 server_base.cc:1034] running on GCE node
I20260926 16:34:40.726397 24287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:34:40.727093 24287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:34:40.728294 24287 hybrid_clock.cc:648] HybridClock initialized: now 1790440480728223 us; error 87 us; skew 500 ppm
Sep 26 16:34:40 dist-test-slave-2zvq krb5kdc[23091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440480, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:34:40.731541 24287 init.cc:377] Logged in from keytab as kudu/127.11.8.194@KRBTEST.COM (short username kudu)
I20260926 16:34:40.732632 24287 webserver.cc:460] Webserver started at http://127.11.8.194:43625/ using document root <none> and password file <none>
I20260926 16:34:40.733232 24287 fs_manager.cc:359] Metadata directory not provided
I20260926 16:34:40.733307 24287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 16:34:40 dist-test-slave-2zvq krb5kdc[23091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440480, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for HTTP/127.11.8.212@KRBTEST.COM
I20260926 16:34:40.752981 24287 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 16:34:40.755631 24305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:40.756978 24287 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:34:40.757074 24287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-1/wal
uuid: "225c06bebce04085b3d1fe0b93d46725"
format_stamp: "Formatted at 2026-09-26 16:34:39 on dist-test-slave-2zvq"
server_key: "134b1499e895b9ccb12c1423862d1da5"
server_key_iv: "053e5c3284cd8fe6c911e40d0fe74d19"
server_key_version: "kuduclusterkey@0"
I20260926 16:34:40.757644 24287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:34:40.784741 24287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:34:40.788544 24287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:34:40.788753 24287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:34:40.789392 24287 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:34:40.790588 24287 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:34:40.790655 24287 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:40.790716 24287 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:34:40.790748 24287 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:40.801494 24287 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.194:43557
I20260926 16:34:40.803370 24418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.194:43557 every 8 connection(s)
I20260926 16:34:40.804083 24287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
I20260926 16:34:40.808754 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 24287
I20260926 16:34:40.809228 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.8.212:54903/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.195:34329
--local_ip_for_outbound_sockets=127.11.8.195
--tserver_master_addrs=127.11.8.254:37397
--webserver_port=45987
--webserver_interface=127.11.8.195
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:34155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.8.212:54903/kms with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 16:34:40 dist-test-slave-2zvq krb5kdc[23091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440480, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:34:40.821877 24288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:40.808398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:44955 (local address 127.11.8.254:37397)
0926 16:34:40.808558 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:40.808563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:40.808900 (+ 337us) server_negotiation.cc:407] Connection header received
0926 16:34:40.810341 (+ 1441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:40.810348 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:40.810415 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:40.810475 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:34:40.812173 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:40.813029 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:40.814150 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:40.814511 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:40.814621 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:40.817813 (+ 3192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:34:40.817817 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:34:40.817820 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:34:40.817850 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:34:40.819479 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:40.820105 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:40.820109 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:40.820111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:40.820200 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:40.820605 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:40.820608 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:40.820610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:40.820833 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:34:40.820902 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:40.821556 (+ 654us) server_negotiation.cc:299] Negotiation successful
0926 16:34:40.821692 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 16:34:40.823660 24421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:40.807306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37397 (local address 127.11.8.194:44955)
0926 16:34:40.808740 (+ 1434us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:40.808797 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 16:34:40.810057 (+ 1260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:34:40.810925 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:34:40.810936 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:34:40.811298 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:34:40.811984 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:40.812007 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:40.813165 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:40.813167 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:40.814023 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:40.814031 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:40.814803 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:40.814809 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:40.815004 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:40.815729 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:34:40.815750 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:34:40.817635 (+ 1885us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:34:40.819619 (+ 1984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:40.819625 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:40.819640 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:40.819980 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:40.820315 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:40.820318 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:40.820320 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:40.820492 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:40.820982 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:34:40.820989 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:34:40.821281 (+ 292us) client_negotiation.cc:769] Sending connection context
0926 16:34:40.822337 (+ 1056us) client_negotiation.cc:240] Negotiation successful
0926 16:34:40.822664 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1213,"thread_start_us":138,"threads_started":1}
I20260926 16:34:40.825223 24419 heartbeater.cc:347] Connected to a master server at 127.11.8.254:37397
I20260926 16:34:40.825462 24419 heartbeater.cc:464] Registering TS with master...
I20260926 16:34:40.826027 24419 heartbeater.cc:511] Master 127.11.8.254:37397 requested a full tablet report, sending...
W20260926 16:34:40.827711 24419 heartbeater.cc:644] Failed to heartbeat to 127.11.8.254:37397 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 16:34:40.827831 24419 heartbeater.cc:464] Registering TS with master...
I20260926 16:34:40.827986 24419 heartbeater.cc:511] Master 127.11.8.254:37397 requested a full tablet report, sending...
I20260926 16:34:40.828814 24419 heartbeater.cc:464] Registering TS with master...
I20260926 16:34:40.828979 24419 heartbeater.cc:511] Master 127.11.8.254:37397 requested a full tablet report, sending...
W20260926 16:34:40.829761 24419 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 16:34:40.955834 24422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:34:40.956113 24422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:34:40.956197 24422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:34:40.960335 24422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:34:40.960412 24422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:34:40.960511 24422 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.195
I20260926 16:34:40.967805 24422 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:34155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.8.212:54903/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.195:34329
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.195
--webserver_port=45987
--tserver_master_addrs=127.11.8.254:37397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24422
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.195
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:34:40.968976 24422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:34:40.970247 24422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:34:40.981984 24428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:34:40.982210 24431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:34:40.981995 24429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:34:40.987742 24422 server_base.cc:1034] running on GCE node
I20260926 16:34:40.988215 24422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:34:40.988933 24422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:34:40.990106 24422 hybrid_clock.cc:648] HybridClock initialized: now 1790440480990077 us; error 41 us; skew 500 ppm
Sep 26 16:34:40 dist-test-slave-2zvq krb5kdc[23091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440480, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:34:40.993048 24422 init.cc:377] Logged in from keytab as kudu/127.11.8.195@KRBTEST.COM (short username kudu)
I20260926 16:34:40.994100 24422 webserver.cc:460] Webserver started at http://127.11.8.195:45987/ using document root <none> and password file <none>
I20260926 16:34:40.994870 24422 fs_manager.cc:359] Metadata directory not provided
I20260926 16:34:40.995016 24422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 16:34:40 dist-test-slave-2zvq krb5kdc[23091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440480, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for HTTP/127.11.8.212@KRBTEST.COM
I20260926 16:34:41.015381 24422 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260926 16:34:41.018585 24439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:41.020345 24422 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 16:34:41.020519 24422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-2/wal
uuid: "2f835f93a0e04a30bdbd13c1586c109d"
format_stamp: "Formatted at 2026-09-26 16:34:39 on dist-test-slave-2zvq"
server_key: "49a666427b8d88b0827b46bf2557db67"
server_key_iv: "61655cfbe4e6ae405824383b8614e8d8"
server_key_version: "kuduclusterkey@0"
I20260926 16:34:41.021313 24422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:34:41.054626 24422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:34:41.058029 24422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:34:41.058236 24422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:34:41.058908 24422 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:34:41.059914 24422 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:34:41.059983 24422 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:41.060047 24422 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:34:41.060102 24422 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:41.070570 24422 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.195:34329
I20260926 16:34:41.071730 24422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
I20260926 16:34:41.072240 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 24422
I20260926 16:34:41.074081 24552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.195:34329 every 8 connection(s)
Sep 26 16:34:41 dist-test-slave-2zvq krb5kdc[23091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440480, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:34:41.085961 24288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:41.073396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60134 (local address 127.11.8.254:37397)
0926 16:34:41.073524 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:41.073528 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:41.074813 (+ 1285us) server_negotiation.cc:407] Connection header received
0926 16:34:41.075013 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:41.075017 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:41.075068 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:41.075118 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:34:41.076851 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:41.077629 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:41.078972 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:41.079306 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:41.079397 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:41.081063 (+ 1666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:34:41.081066 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:34:41.081070 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:34:41.081095 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:34:41.082936 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:41.084642 (+ 1706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:41.084648 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:41.084650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:41.084739 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:41.085048 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:41.085054 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:41.085058 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:41.085287 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:34:41.085495 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:41.085640 (+ 145us) server_negotiation.cc:299] Negotiation successful
0926 16:34:41.085770 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 16:34:41.092145 24557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:41.076249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37397 (local address 127.11.8.195:53407)
0926 16:34:41.077359 (+ 1110us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:41.077408 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 16:34:41.078418 (+ 1010us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:34:41.079931 (+ 1513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:34:41.079940 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:34:41.080220 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:34:41.080810 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:41.080833 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:41.081942 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:41.081949 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:41.083312 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:41.083321 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:41.083819 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:41.083824 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:41.084004 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:41.084552 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:34:41.084568 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:34:41.086671 (+ 2103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:34:41.088858 (+ 2187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:41.088864 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:41.088880 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:41.089235 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:41.089553 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:41.089557 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:41.089559 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:41.089742 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:41.090207 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:34:41.090214 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:34:41.090541 (+ 327us) client_negotiation.cc:769] Sending connection context
0926 16:34:41.090741 (+ 200us) client_negotiation.cc:240] Negotiation successful
0926 16:34:41.091067 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":912,"thread_start_us":126,"threads_started":1}
I20260926 16:34:41.092988 24556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:41.075703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:53407 (local address 127.11.8.254:37397)
0926 16:34:41.079090 (+ 3387us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:41.079093 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:41.079104 (+ 11us) server_negotiation.cc:407] Connection header received
0926 16:34:41.079133 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:41.079136 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:41.079176 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:41.079298 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:34:41.081010 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:41.081787 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:41.083439 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:41.083667 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:41.083781 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:41.087003 (+ 3222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:34:41.087006 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:34:41.087010 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:34:41.087036 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:34:41.088740 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:41.089354 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:41.089358 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:41.089359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:41.089450 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:41.089857 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:41.089863 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:41.089864 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:41.090063 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:34:41.090156 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:41.092676 (+ 2520us) server_negotiation.cc:299] Negotiation successful
0926 16:34:41.092824 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3322,"thread_start_us":82,"threads_started":1}
I20260926 16:34:41.093906 24553 heartbeater.cc:347] Connected to a master server at 127.11.8.254:37397
I20260926 16:34:41.094157 24553 heartbeater.cc:464] Registering TS with master...
I20260926 16:34:41.094776 24553 heartbeater.cc:511] Master 127.11.8.254:37397 requested a full tablet report, sending...
W20260926 16:34:41.096311 24553 heartbeater.cc:644] Failed to heartbeat to 127.11.8.254:37397 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 16:34:41.096441 24553 heartbeater.cc:464] Registering TS with master...
I20260926 16:34:41.096608 24553 heartbeater.cc:511] Master 127.11.8.254:37397 requested a full tablet report, sending...
I20260926 16:34:41.097396 24553 heartbeater.cc:464] Registering TS with master...
I20260926 16:34:41.097561 24553 heartbeater.cc:511] Master 127.11.8.254:37397 requested a full tablet report, sending...
W20260926 16:34:41.099171 24553 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 26 16:34:41 dist-test-slave-2zvq krb5kdc[23091](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790440481, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:34:41.603825 24285 heartbeater.cc:464] Registering TS with master...
I20260926 16:34:41.604152 24285 heartbeater.cc:511] Master 127.11.8.254:37397 requested a full tablet report, sending...
I20260926 16:34:41.605333 24087 ts_manager.cc:194] Registered new tserver with Master: 349e20badcc1476b905b8c4cc0119e24 (127.11.8.193:33721)
I20260926 16:34:41.607118 24087 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.193@KRBTEST.COM'} at 127.11.8.193:39915
I20260926 16:34:41.829977 24419 heartbeater.cc:464] Registering TS with master...
I20260926 16:34:41.830283 24419 heartbeater.cc:511] Master 127.11.8.254:37397 requested a full tablet report, sending...
I20260926 16:34:41.830924 24087 ts_manager.cc:194] Registered new tserver with Master: 225c06bebce04085b3d1fe0b93d46725 (127.11.8.194:43557)
I20260926 16:34:41.831720 24087 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.194@KRBTEST.COM'} at 127.11.8.194:44955
Sep 26 16:34:41 dist-test-slave-2zvq krb5kdc[23091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440481, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for HTTP/127.11.8.212@KRBTEST.COM
Sep 26 16:34:42 dist-test-slave-2zvq krb5kdc[23091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440481, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for HTTP/127.11.8.212@KRBTEST.COM
I20260926 16:34:42.099401 24553 heartbeater.cc:464] Registering TS with master...
I20260926 16:34:42.099712 24553 heartbeater.cc:511] Master 127.11.8.254:37397 requested a full tablet report, sending...
I20260926 16:34:42.101082 24087 ts_manager.cc:194] Registered new tserver with Master: 2f835f93a0e04a30bdbd13c1586c109d (127.11.8.195:34329)
I20260926 16:34:42.101991 24087 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.195@KRBTEST.COM'} at 127.11.8.195:53407
I20260926 16:34:42.103413 11299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 16:34:42.125198 24575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:42.107589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60150 (local address 127.11.8.254:37397)
0926 16:34:42.107838 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:42.107844 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:42.108181 (+ 337us) server_negotiation.cc:407] Connection header received
0926 16:34:42.108725 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:42.108729 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:42.108779 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:42.108839 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:34:42.112338 (+ 3499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:42.113017 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:42.115190 (+ 2173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:42.115459 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:42.115516 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:42.118957 (+ 3441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:34:42.118961 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:34:42.118964 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:34:42.118992 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:34:42.121065 (+ 2073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:42.122730 (+ 1665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:42.122734 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:42.122736 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:42.122830 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:42.123456 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:42.123460 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:42.123462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:42.123682 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:34:42.123801 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:42.124861 (+ 1060us) server_negotiation.cc:299] Negotiation successful
0926 16:34:42.125002 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":108,"threads_started":1}
I20260926 16:34:42.130997 24087 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60150:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:34:42.136152 24087 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:34:42.163595 24584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:42.147221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:33721 (local address 127.0.0.1:47490)
0926 16:34:42.154764 (+ 7543us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:42.154820 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0926 16:34:42.154964 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:34:42.155913 (+ 949us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:34:42.155920 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:34:42.155942 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:34:42.156076 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:42.156083 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:42.157470 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:42.157474 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:42.160585 (+ 3111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:42.161252 (+ 667us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:42.163022 (+ 1770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:42.163026 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:42.163155 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:42.163205 (+ 50us) client_negotiation.cc:769] Sending connection context
0926 16:34:42.163306 (+ 101us) client_negotiation.cc:240] Negotiation successful
0926 16:34:42.163426 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7355,"thread_start_us":105,"threads_started":1}
I20260926 16:34:42.164635 24583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:42.147376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47490 (local address 127.11.8.193:33721)
0926 16:34:42.155268 (+ 7892us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:42.155275 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:42.155300 (+ 25us) server_negotiation.cc:407] Connection header received
0926 16:34:42.155376 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:42.155382 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:42.155595 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:42.155742 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:34:42.156223 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:42.157351 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:42.161384 (+ 4033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:42.162224 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:42.162429 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:42.162594 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:42.164234 (+ 1640us) server_negotiation.cc:299] Negotiation successful
0926 16:34:42.164485 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7779,"thread_start_us":66,"threads_started":1}
I20260926 16:34:42.168094 24218 tablet_service.cc:1467] Processing CreateTablet for tablet d29000c57fe2431fb37563a3a3f1d8af (DEFAULT_TABLE table=test-table [id=bf044e2c12544f61b21fa70ac975614c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:34:42.169345 24218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d29000c57fe2431fb37563a3a3f1d8af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:34:42.179679 24582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:42.147590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:34329 (local address 127.0.0.1:36778)
0926 16:34:42.157564 (+ 9974us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:42.157577 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:34:42.157662 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:34:42.159903 (+ 2241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:34:42.159906 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:34:42.159923 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:34:42.160001 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:42.160007 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:42.164700 (+ 4693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:42.164703 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:42.177571 (+ 12868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:42.177586 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:42.179136 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:42.179140 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:42.179271 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:42.179312 (+ 41us) client_negotiation.cc:769] Sending connection context
0926 16:34:42.179393 (+ 81us) client_negotiation.cc:240] Negotiation successful
0926 16:34:42.179485 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9913,"mutex_wait_us":6,"thread_start_us":84,"threads_started":1}
I20260926 16:34:42.180218 24585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:42.147829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36778 (local address 127.11.8.195:34329)
0926 16:34:42.154485 (+ 6656us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:42.154493 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:42.157658 (+ 3165us) server_negotiation.cc:407] Connection header received
0926 16:34:42.157899 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:42.157908 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:42.159221 (+ 1313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:42.159749 (+ 528us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:34:42.160132 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:42.164553 (+ 4421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:42.177765 (+ 13212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:42.178698 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:42.178848 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:42.179056 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:42.179949 (+ 893us) server_negotiation.cc:299] Negotiation successful
0926 16:34:42.180064 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6532,"thread_start_us":58,"threads_started":1}
I20260926 16:34:42.183790 24478 tablet_service.cc:1467] Processing CreateTablet for tablet d29000c57fe2431fb37563a3a3f1d8af (DEFAULT_TABLE table=test-table [id=bf044e2c12544f61b21fa70ac975614c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:34:42.184624 24586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:42.147951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:43557 (local address 127.0.0.1:48572)
0926 16:34:42.157868 (+ 9917us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:42.157880 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:34:42.157948 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:34:42.179195 (+ 21247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:34:42.179198 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:34:42.179226 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:34:42.179396 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:42.179403 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:42.181703 (+ 2300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:42.181706 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:42.182937 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:42.182949 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:42.184225 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:42.184229 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:42.184325 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:42.184350 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 16:34:42.184414 (+ 64us) client_negotiation.cc:240] Negotiation successful
0926 16:34:42.184484 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9865,"mutex_wait_us":32,"thread_start_us":76,"threads_started":1}
I20260926 16:34:42.184962 24478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d29000c57fe2431fb37563a3a3f1d8af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:34:42.185153 24587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:42.148101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48572 (local address 127.11.8.194:43557)
0926 16:34:42.158195 (+ 10094us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:42.158202 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:42.158223 (+ 21us) server_negotiation.cc:407] Connection header received
0926 16:34:42.158316 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:42.158321 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:42.158502 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:42.158641 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:34:42.179570 (+ 20929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:42.181556 (+ 1986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:42.183090 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:42.183902 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:42.184008 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:42.184159 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:42.184898 (+ 739us) server_negotiation.cc:299] Negotiation successful
0926 16:34:42.185001 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9974,"thread_start_us":104,"threads_started":1}
I20260926 16:34:42.187960 24335 tablet_service.cc:1467] Processing CreateTablet for tablet d29000c57fe2431fb37563a3a3f1d8af (DEFAULT_TABLE table=test-table [id=bf044e2c12544f61b21fa70ac975614c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:34:42.188992 24335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d29000c57fe2431fb37563a3a3f1d8af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:34:42.197890 24588 tablet_bootstrap.cc:492] T d29000c57fe2431fb37563a3a3f1d8af P 349e20badcc1476b905b8c4cc0119e24: Bootstrap starting.
I20260926 16:34:42.200362 24588 tablet_bootstrap.cc:654] T d29000c57fe2431fb37563a3a3f1d8af P 349e20badcc1476b905b8c4cc0119e24: Neither blocks nor log segments found. Creating new log.
I20260926 16:34:42.201239 24588 log.cc:824] T d29000c57fe2431fb37563a3a3f1d8af P 349e20badcc1476b905b8c4cc0119e24: Log is configured to *not* fsync() on all Append() calls
I20260926 16:34:42.201982 24589 tablet_bootstrap.cc:492] T d29000c57fe2431fb37563a3a3f1d8af P 225c06bebce04085b3d1fe0b93d46725: Bootstrap starting.
I20260926 16:34:42.204077 24589 tablet_bootstrap.cc:654] T d29000c57fe2431fb37563a3a3f1d8af P 225c06bebce04085b3d1fe0b93d46725: Neither blocks nor log segments found. Creating new log.
I20260926 16:34:42.204854 24589 log.cc:824] T d29000c57fe2431fb37563a3a3f1d8af P 225c06bebce04085b3d1fe0b93d46725: Log is configured to *not* fsync() on all Append() calls
I20260926 16:34:42.210932 24591 tablet_bootstrap.cc:492] T d29000c57fe2431fb37563a3a3f1d8af P 2f835f93a0e04a30bdbd13c1586c109d: Bootstrap starting.
I20260926 16:34:42.213178 24591 tablet_bootstrap.cc:654] T d29000c57fe2431fb37563a3a3f1d8af P 2f835f93a0e04a30bdbd13c1586c109d: Neither blocks nor log segments found. Creating new log.
I20260926 16:34:42.214112 24591 log.cc:824] T d29000c57fe2431fb37563a3a3f1d8af P 2f835f93a0e04a30bdbd13c1586c109d: Log is configured to *not* fsync() on all Append() calls
I20260926 16:34:42.216557 24588 tablet_bootstrap.cc:492] T d29000c57fe2431fb37563a3a3f1d8af P 349e20badcc1476b905b8c4cc0119e24: No bootstrap required, opened a new log
I20260926 16:34:42.216771 24588 ts_tablet_manager.cc:1397] T d29000c57fe2431fb37563a3a3f1d8af P 349e20badcc1476b905b8c4cc0119e24: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.000s
I20260926 16:34:42.220711 24588 raft_consensus.cc:348] T d29000c57fe2431fb37563a3a3f1d8af P 349e20badcc1476b905b8c4cc0119e24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349e20badcc1476b905b8c4cc0119e24" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33721 } } peers { permanent_uuid: "225c06bebce04085b3d1fe0b93d46725" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43557 } } peers { permanent_uuid: "2f835f93a0e04a30bdbd13c1586c109d" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 34329 } }
I20260926 16:34:42.220992 24588 raft_consensus.cc:374] T d29000c57fe2431fb37563a3a3f1d8af P 349e20badcc1476b905b8c4cc0119e24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:34:42.221050 24588 raft_consensus.cc:729] T d29000c57fe2431fb37563a3a3f1d8af P 349e20badcc1476b905b8c4cc0119e24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 349e20badcc1476b905b8c4cc0119e24, State: Initialized, Role: FOLLOWER
I20260926 16:34:42.223399 24589 tablet_bootstrap.cc:492] T d29000c57fe2431fb37563a3a3f1d8af P 225c06bebce04085b3d1fe0b93d46725: No bootstrap required, opened a new log
I20260926 16:34:42.223613 24589 ts_tablet_manager.cc:1397] T d29000c57fe2431fb37563a3a3f1d8af P 225c06bebce04085b3d1fe0b93d46725: Time spent bootstrapping tablet: real 0.022s user 0.004s sys 0.000s
I20260926 16:34:42.223623 24591 tablet_bootstrap.cc:492] T d29000c57fe2431fb37563a3a3f1d8af P 2f835f93a0e04a30bdbd13c1586c109d: No bootstrap required, opened a new log
I20260926 16:34:42.223819 24591 ts_tablet_manager.cc:1397] T d29000c57fe2431fb37563a3a3f1d8af P 2f835f93a0e04a30bdbd13c1586c109d: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.000s
I20260926 16:34:42.227476 24589 raft_consensus.cc:348] T d29000c57fe2431fb37563a3a3f1d8af P 225c06bebce04085b3d1fe0b93d46725 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349e20badcc1476b905b8c4cc0119e24" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33721 } } peers { permanent_uuid: "225c06bebce04085b3d1fe0b93d46725" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43557 } } peers { permanent_uuid: "2f835f93a0e04a30bdbd13c1586c109d" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 34329 } }
I20260926 16:34:42.227738 24589 raft_consensus.cc:374] T d29000c57fe2431fb37563a3a3f1d8af P 225c06bebce04085b3d1fe0b93d46725 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:34:42.227794 24589 raft_consensus.cc:729] T d29000c57fe2431fb37563a3a3f1d8af P 225c06bebce04085b3d1fe0b93d46725 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 225c06bebce04085b3d1fe0b93d46725, State: Initialized, Role: FOLLOWER
I20260926 16:34:42.228320 24588 consensus_queue.cc:260] T d29000c57fe2431fb37563a3a3f1d8af P 349e20badcc1476b905b8c4cc0119e24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349e20badcc1476b905b8c4cc0119e24" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33721 } } peers { permanent_uuid: "225c06bebce04085b3d1fe0b93d46725" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43557 } } peers { permanent_uuid: "2f835f93a0e04a30bdbd13c1586c109d" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 34329 } }
I20260926 16:34:42.229357 24588 ts_tablet_manager.cc:1428] T d29000c57fe2431fb37563a3a3f1d8af P 349e20badcc1476b905b8c4cc0119e24: Time spent starting tablet: real 0.013s user 0.007s sys 0.000s
I20260926 16:34:42.231551 24591 raft_consensus.cc:348] T d29000c57fe2431fb37563a3a3f1d8af P 2f835f93a0e04a30bdbd13c1586c109d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349e20badcc1476b905b8c4cc0119e24" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33721 } } peers { permanent_uuid: "225c06bebce04085b3d1fe0b93d46725" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43557 } } peers { permanent_uuid: "2f835f93a0e04a30bdbd13c1586c109d" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 34329 } }
I20260926 16:34:42.231827 24591 raft_consensus.cc:374] T d29000c57fe2431fb37563a3a3f1d8af P 2f835f93a0e04a30bdbd13c1586c109d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:34:42.231899 24591 raft_consensus.cc:729] T d29000c57fe2431fb37563a3a3f1d8af P 2f835f93a0e04a30bdbd13c1586c109d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f835f93a0e04a30bdbd13c1586c109d, State: Initialized, Role: FOLLOWER
I20260926 16:34:42.234544 24591 consensus_queue.cc:260] T d29000c57fe2431fb37563a3a3f1d8af P 2f835f93a0e04a30bdbd13c1586c109d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349e20badcc1476b905b8c4cc0119e24" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33721 } } peers { permanent_uuid: "225c06bebce04085b3d1fe0b93d46725" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43557 } } peers { permanent_uuid: "2f835f93a0e04a30bdbd13c1586c109d" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 34329 } }
I20260926 16:34:42.234550 24589 consensus_queue.cc:260] T d29000c57fe2431fb37563a3a3f1d8af P 225c06bebce04085b3d1fe0b93d46725 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349e20badcc1476b905b8c4cc0119e24" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33721 } } peers { permanent_uuid: "225c06bebce04085b3d1fe0b93d46725" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43557 } } peers { permanent_uuid: "2f835f93a0e04a30bdbd13c1586c109d" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 34329 } }
I20260926 16:34:42.235321 24419 heartbeater.cc:503] Master 127.11.8.254:37397 was elected leader, sending a full tablet report...
I20260926 16:34:42.235560 24589 ts_tablet_manager.cc:1428] T d29000c57fe2431fb37563a3a3f1d8af P 225c06bebce04085b3d1fe0b93d46725: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
I20260926 16:34:42.235594 24591 ts_tablet_manager.cc:1428] T d29000c57fe2431fb37563a3a3f1d8af P 2f835f93a0e04a30bdbd13c1586c109d: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
I20260926 16:34:42.235771 24285 heartbeater.cc:503] Master 127.11.8.254:37397 was elected leader, sending a full tablet report...
I20260926 16:34:42.235937 24553 heartbeater.cc:503] Master 127.11.8.254:37397 was elected leader, sending a full tablet report...
I20260926 16:34:42.300109 24594 raft_consensus.cc:482] T d29000c57fe2431fb37563a3a3f1d8af P 349e20badcc1476b905b8c4cc0119e24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:34:42.300352 24594 raft_consensus.cc:504] T d29000c57fe2431fb37563a3a3f1d8af P 349e20badcc1476b905b8c4cc0119e24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349e20badcc1476b905b8c4cc0119e24" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33721 } } peers { permanent_uuid: "225c06bebce04085b3d1fe0b93d46725" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43557 } } peers { permanent_uuid: "2f835f93a0e04a30bdbd13c1586c109d" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 34329 } }
I20260926 16:34:42.301703 24594 leader_election.cc:288] T d29000c57fe2431fb37563a3a3f1d8af P 349e20badcc1476b905b8c4cc0119e24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 225c06bebce04085b3d1fe0b93d46725 (127.11.8.194:43557), 2f835f93a0e04a30bdbd13c1586c109d (127.11.8.195:34329)
W20260926 16:34:42.306408 24420 tablet.cc:2369] T d29000c57fe2431fb37563a3a3f1d8af P 225c06bebce04085b3d1fe0b93d46725: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 16:34:42.311856 24286 tablet.cc:2369] T d29000c57fe2431fb37563a3a3f1d8af P 349e20badcc1476b905b8c4cc0119e24: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:34:42.322602 24585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:42.308422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:44763 (local address 127.11.8.195:34329)
0926 16:34:42.308552 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:42.308557 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:42.315001 (+ 6444us) server_negotiation.cc:407] Connection header received
0926 16:34:42.315214 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:42.315218 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:42.315266 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:42.315313 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:34:42.315721 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:42.316505 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:42.318997 (+ 2492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:42.319638 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:42.319734 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:42.319785 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:42.322385 (+ 2600us) server_negotiation.cc:299] Negotiation successful
0926 16:34:42.322472 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 16:34:42.323064 24597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:42.308330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:34329 (local address 127.11.8.193:44763)
0926 16:34:42.314907 (+ 6577us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:42.314922 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:34:42.315080 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:34:42.315446 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:34:42.315449 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:34:42.315465 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:34:42.315601 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:42.315608 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:42.316646 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:42.316649 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:42.317961 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:42.317972 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:42.320792 (+ 2820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:42.320796 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:42.320907 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:42.320931 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 16:34:42.322826 (+ 1895us) client_negotiation.cc:240] Negotiation successful
0926 16:34:42.322901 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6500,"thread_start_us":121,"threads_started":1}
I20260926 16:34:42.323753 24507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d29000c57fe2431fb37563a3a3f1d8af" candidate_uuid: "349e20badcc1476b905b8c4cc0119e24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f835f93a0e04a30bdbd13c1586c109d" is_pre_election: true
I20260926 16:34:42.324092 24507 raft_consensus.cc:2454] T d29000c57fe2431fb37563a3a3f1d8af P 2f835f93a0e04a30bdbd13c1586c109d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 349e20badcc1476b905b8c4cc0119e24 in term 0.
I20260926 16:34:42.324503 24587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:42.308891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:39689 (local address 127.11.8.194:43557)
0926 16:34:42.309000 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:42.309004 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:42.318326 (+ 9322us) server_negotiation.cc:407] Connection header received
0926 16:34:42.318521 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:42.318525 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:42.318580 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:42.318627 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:34:42.319869 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:42.320637 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:42.323261 (+ 2624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:42.323827 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:42.323917 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:42.323961 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:42.324321 (+ 360us) server_negotiation.cc:299] Negotiation successful
0926 16:34:42.324398 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260926 16:34:42.324671 24175 leader_election.cc:302] T d29000c57fe2431fb37563a3a3f1d8af P 349e20badcc1476b905b8c4cc0119e24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f835f93a0e04a30bdbd13c1586c109d, 349e20badcc1476b905b8c4cc0119e24; no voters:
I20260926 16:34:42.324867 24598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:42.308810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:43557 (local address 127.11.8.193:39689)
0926 16:34:42.318212 (+ 9402us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:42.318231 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 16:34:42.318397 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:34:42.318752 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:34:42.318755 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:34:42.318771 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:34:42.318864 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:42.318870 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:42.321057 (+ 2187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:42.321060 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:42.322207 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:42.322218 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:42.324058 (+ 1840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:42.324063 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:42.324164 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:42.324192 (+ 28us) client_negotiation.cc:769] Sending connection context
0926 16:34:42.324679 (+ 487us) client_negotiation.cc:240] Negotiation successful
0926 16:34:42.324745 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9330,"thread_start_us":102,"threads_started":1}
I20260926 16:34:42.325013 24594 raft_consensus.cc:2781] T d29000c57fe2431fb37563a3a3f1d8af P 349e20badcc1476b905b8c4cc0119e24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:34:42.325111 24594 raft_consensus.cc:482] T d29000c57fe2431fb37563a3a3f1d8af P 349e20badcc1476b905b8c4cc0119e24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:34:42.325165 24594 raft_consensus.cc:3037] T d29000c57fe2431fb37563a3a3f1d8af P 349e20badcc1476b905b8c4cc0119e24 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:34:42.325745 24373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d29000c57fe2431fb37563a3a3f1d8af" candidate_uuid: "349e20badcc1476b905b8c4cc0119e24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "225c06bebce04085b3d1fe0b93d46725" is_pre_election: true
I20260926 16:34:42.326042 24373 raft_consensus.cc:2454] T d29000c57fe2431fb37563a3a3f1d8af P 225c06bebce04085b3d1fe0b93d46725 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 349e20badcc1476b905b8c4cc0119e24 in term 0.
I20260926 16:34:42.326644 24594 raft_consensus.cc:504] T d29000c57fe2431fb37563a3a3f1d8af P 349e20badcc1476b905b8c4cc0119e24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349e20badcc1476b905b8c4cc0119e24" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33721 } } peers { permanent_uuid: "225c06bebce04085b3d1fe0b93d46725" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43557 } } peers { permanent_uuid: "2f835f93a0e04a30bdbd13c1586c109d" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 34329 } }
I20260926 16:34:42.327039 24594 leader_election.cc:288] T d29000c57fe2431fb37563a3a3f1d8af P 349e20badcc1476b905b8c4cc0119e24 [CANDIDATE]: Term 1 election: Requested vote from peers 225c06bebce04085b3d1fe0b93d46725 (127.11.8.194:43557), 2f835f93a0e04a30bdbd13c1586c109d (127.11.8.195:34329)
I20260926 16:34:42.327533 24373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d29000c57fe2431fb37563a3a3f1d8af" candidate_uuid: "349e20badcc1476b905b8c4cc0119e24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "225c06bebce04085b3d1fe0b93d46725"
W20260926 16:34:42.327644 24554 tablet.cc:2369] T d29000c57fe2431fb37563a3a3f1d8af P 2f835f93a0e04a30bdbd13c1586c109d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:34:42.327666 24373 raft_consensus.cc:3037] T d29000c57fe2431fb37563a3a3f1d8af P 225c06bebce04085b3d1fe0b93d46725 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:34:42.328949 24373 raft_consensus.cc:2454] T d29000c57fe2431fb37563a3a3f1d8af P 225c06bebce04085b3d1fe0b93d46725 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 349e20badcc1476b905b8c4cc0119e24 in term 1.
I20260926 16:34:42.329275 24174 leader_election.cc:302] T d29000c57fe2431fb37563a3a3f1d8af P 349e20badcc1476b905b8c4cc0119e24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 225c06bebce04085b3d1fe0b93d46725, 349e20badcc1476b905b8c4cc0119e24; no voters:
I20260926 16:34:42.329725 24507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d29000c57fe2431fb37563a3a3f1d8af" candidate_uuid: "349e20badcc1476b905b8c4cc0119e24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f835f93a0e04a30bdbd13c1586c109d"
I20260926 16:34:42.329851 24507 raft_consensus.cc:3037] T d29000c57fe2431fb37563a3a3f1d8af P 2f835f93a0e04a30bdbd13c1586c109d [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:34:42.331104 24507 raft_consensus.cc:2454] T d29000c57fe2431fb37563a3a3f1d8af P 2f835f93a0e04a30bdbd13c1586c109d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 349e20badcc1476b905b8c4cc0119e24 in term 1.
I20260926 16:34:42.331521 24594 raft_consensus.cc:2781] T d29000c57fe2431fb37563a3a3f1d8af P 349e20badcc1476b905b8c4cc0119e24 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:34:42.331775 24594 raft_consensus.cc:686] T d29000c57fe2431fb37563a3a3f1d8af P 349e20badcc1476b905b8c4cc0119e24 [term 1 LEADER]: Becoming Leader. State: Replica: 349e20badcc1476b905b8c4cc0119e24, State: Running, Role: LEADER
I20260926 16:34:42.332137 24594 consensus_queue.cc:237] T d29000c57fe2431fb37563a3a3f1d8af P 349e20badcc1476b905b8c4cc0119e24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349e20badcc1476b905b8c4cc0119e24" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33721 } } peers { permanent_uuid: "225c06bebce04085b3d1fe0b93d46725" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43557 } } peers { permanent_uuid: "2f835f93a0e04a30bdbd13c1586c109d" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 34329 } }
I20260926 16:34:42.335566 24086 catalog_manager.cc:5495] T d29000c57fe2431fb37563a3a3f1d8af P 349e20badcc1476b905b8c4cc0119e24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 349e20badcc1476b905b8c4cc0119e24 (127.11.8.193). New cstate: current_term: 1 leader_uuid: "349e20badcc1476b905b8c4cc0119e24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "349e20badcc1476b905b8c4cc0119e24" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 33721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "225c06bebce04085b3d1fe0b93d46725" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f835f93a0e04a30bdbd13c1586c109d" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 34329 } health_report { overall_health: UNKNOWN } } }
I20260926 16:34:42.357257 24583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:42.352955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47496 (local address 127.11.8.193:33721)
0926 16:34:42.353184 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:42.353188 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:42.353203 (+ 15us) server_negotiation.cc:407] Connection header received
0926 16:34:42.353282 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:42.353285 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:42.353354 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:42.353456 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:34:42.353887 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:42.354698 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:42.355763 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:42.356018 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:42.356135 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:42.356438 (+ 303us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:34:42.356848 (+ 410us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:34:42.356985 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:42.357027 (+ 42us) server_negotiation.cc:299] Negotiation successful
0926 16:34:42.357081 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139}
I20260926 16:34:42.364286 24373 raft_consensus.cc:1261] T d29000c57fe2431fb37563a3a3f1d8af P 225c06bebce04085b3d1fe0b93d46725 [term 1 FOLLOWER]: Refusing update from remote peer 349e20badcc1476b905b8c4cc0119e24: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:34:42.364286 24507 raft_consensus.cc:1261] T d29000c57fe2431fb37563a3a3f1d8af P 2f835f93a0e04a30bdbd13c1586c109d [term 1 FOLLOWER]: Refusing update from remote peer 349e20badcc1476b905b8c4cc0119e24: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:34:42.364995 24594 consensus_queue.cc:1035] T d29000c57fe2431fb37563a3a3f1d8af P 349e20badcc1476b905b8c4cc0119e24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f835f93a0e04a30bdbd13c1586c109d" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 34329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:34:42.365352 24599 consensus_queue.cc:1035] T d29000c57fe2431fb37563a3a3f1d8af P 349e20badcc1476b905b8c4cc0119e24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "225c06bebce04085b3d1fe0b93d46725" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 43557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:34:42.372445 24604 mvcc.cc:205] Tried to move back new op lower bound from 7333644215754076160 to 7333644215635521536. Current Snapshot: MvccSnapshot[applied={T|T < 7333644215754076160}]
I20260926 16:34:42.377228 24611 mvcc.cc:205] Tried to move back new op lower bound from 7333644215754076160 to 7333644215635521536. Current Snapshot: MvccSnapshot[applied={T|T < 7333644215754076160}]
I20260926 16:34:42.379485 24610 mvcc.cc:205] Tried to move back new op lower bound from 7333644215754076160 to 7333644215635521536. Current Snapshot: MvccSnapshot[applied={T|T < 7333644215754076160}]
I20260926 16:34:42.387619 24087 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:34:42.387811 24087 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:34:42.390516 24087 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 309a8bd2d36144d78a6db0864f78409e: Sending DeleteTablet for 3 replicas of tablet d29000c57fe2431fb37563a3a3f1d8af
I20260926 16:34:42.391386 24218 tablet_service.cc:1514] Processing DeleteTablet for tablet d29000c57fe2431fb37563a3a3f1d8af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:34:42 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:47490
I20260926 16:34:42.391428 24335 tablet_service.cc:1514] Processing DeleteTablet for tablet d29000c57fe2431fb37563a3a3f1d8af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:34:42 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:48572
I20260926 16:34:42.391594 24478 tablet_service.cc:1514] Processing DeleteTablet for tablet d29000c57fe2431fb37563a3a3f1d8af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:34:42 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:36778
I20260926 16:34:42.392078 24621 tablet_replica.cc:330] T d29000c57fe2431fb37563a3a3f1d8af P 2f835f93a0e04a30bdbd13c1586c109d: stopping tablet replica
I20260926 16:34:42.392436 24621 raft_consensus.cc:2229] T d29000c57fe2431fb37563a3a3f1d8af P 2f835f93a0e04a30bdbd13c1586c109d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:34:42.392452 24619 tablet_replica.cc:330] T d29000c57fe2431fb37563a3a3f1d8af P 225c06bebce04085b3d1fe0b93d46725: stopping tablet replica
I20260926 16:34:42.392825 24619 raft_consensus.cc:2229] T d29000c57fe2431fb37563a3a3f1d8af P 225c06bebce04085b3d1fe0b93d46725 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:34:42.392860 24621 raft_consensus.cc:2258] T d29000c57fe2431fb37563a3a3f1d8af P 2f835f93a0e04a30bdbd13c1586c109d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:34:42.392946 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 24125
I20260926 16:34:42.393573 24619 raft_consensus.cc:2258] T d29000c57fe2431fb37563a3a3f1d8af P 225c06bebce04085b3d1fe0b93d46725 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:34:42.395038 24621 ts_tablet_manager.cc:1905] T d29000c57fe2431fb37563a3a3f1d8af P 2f835f93a0e04a30bdbd13c1586c109d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:34:42.396853 24619 ts_tablet_manager.cc:1905] T d29000c57fe2431fb37563a3a3f1d8af P 225c06bebce04085b3d1fe0b93d46725: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:34:42.398526 24621 ts_tablet_manager.cc:1918] T d29000c57fe2431fb37563a3a3f1d8af P 2f835f93a0e04a30bdbd13c1586c109d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:34:42.398629 24621 log.cc:1196] T d29000c57fe2431fb37563a3a3f1d8af P 2f835f93a0e04a30bdbd13c1586c109d: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-2/wal/wals/d29000c57fe2431fb37563a3a3f1d8af
I20260926 16:34:42.398969 24621 ts_tablet_manager.cc:1939] T d29000c57fe2431fb37563a3a3f1d8af P 2f835f93a0e04a30bdbd13c1586c109d: Deleting consensus metadata
I20260926 16:34:42.399760 24619 ts_tablet_manager.cc:1918] T d29000c57fe2431fb37563a3a3f1d8af P 225c06bebce04085b3d1fe0b93d46725: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:34:42.399863 24619 log.cc:1196] T d29000c57fe2431fb37563a3a3f1d8af P 225c06bebce04085b3d1fe0b93d46725: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790440324823152-11299-0/minicluster-data/ts-1/wal/wals/d29000c57fe2431fb37563a3a3f1d8af
I20260926 16:34:42.400228 24619 ts_tablet_manager.cc:1939] T d29000c57fe2431fb37563a3a3f1d8af P 225c06bebce04085b3d1fe0b93d46725: Deleting consensus metadata
I20260926 16:34:42.400285 24074 catalog_manager.cc:4841] TS 2f835f93a0e04a30bdbd13c1586c109d (127.11.8.195:34329): tablet d29000c57fe2431fb37563a3a3f1d8af (table test-table [id=bf044e2c12544f61b21fa70ac975614c]) successfully deleted
W20260926 16:34:42.400789 24073 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.8.193:33721 (error 108)
I20260926 16:34:42.401144 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 24287
I20260926 16:34:42.403402 24586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:42.403066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:33721 (local address 127.0.0.1:47502)
0926 16:34:42.403220 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:42.403299 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.193:33721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260926 16:34:42.403574 24073 catalog_manager.cc:4584] TS 349e20badcc1476b905b8c4cc0119e24 (127.11.8.193:33721): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d29000c57fe2431fb37563a3a3f1d8af: Network error: Client connection negotiation failed: client connection to 127.11.8.193:33721: connect: Connection refused (error 111)
I20260926 16:34:42.407665 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 24422
I20260926 16:34:42.409336 24586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:42.407793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:43557 (local address 127.0.0.1:48588)
0926 16:34:42.409214 (+ 1421us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:42.409264 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.194:43557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1345}
I20260926 16:34:42.413355 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 24054
I20260926 16:34:42.421584 11299 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260926 16:34:42.756242 11299 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260926 16:34:42.756376 11299 mini_ranger.cc:66] Stopping Ranger...
I20260926 16:34:43.091866 11299 mini_ranger.cc:68] Stopped Ranger
2026-09-26 16:34:43.092 UTC [23121] LOG: received smart shutdown request
2026-09-26 16:34:43.095 UTC [23121] LOG: background worker "logical replication launcher" (PID 23130) exited with exit code 1
2026-09-26 16:34:43.096 UTC [23125] LOG: shutting down
2026-09-26 16:34:43.254 UTC [23121] LOG: database system is shut down
2026-09-26T16:34:43Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (47076 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:34:43 dist-test-slave-2zvq krb5kdc[24631](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:34:43 dist-test-slave-2zvq krb5kdc[24631](info): set up 2 sockets
Sep 26 16:34:43 dist-test-slave-2zvq krb5kdc[24631](info): commencing operation
krb5kdc: starting...
W20260926 16:34:44.420136 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:34:44 dist-test-slave-2zvq krb5kdc[24631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:34:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:34:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
I20260926 16:34:44.536934 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:34705
--webserver_interface=127.11.8.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:43851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:34705,127.11.8.253:46735,127.11.8.252:45237
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:34:44.647192 24644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:34:44.647480 24644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:34:44.647552 24644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:34:44.651525 24644 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:34:44.651610 24644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:34:44.651645 24644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:34:44.651684 24644 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:34:44.651722 24644 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:34:44.657064 24644 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:43851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:34705,127.11.8.253:46735,127.11.8.252:45237
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:34705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:34:44.658092 24644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:34:44.659198 24644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:34:44.667564 24649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:34:44.667593 24652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:34:44.667595 24650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:34:44.667642 24644 server_base.cc:1034] running on GCE node
I20260926 16:34:44.668430 24644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:34:44.669420 24644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:34:44.670572 24644 hybrid_clock.cc:648] HybridClock initialized: now 1790440484670560 us; error 24 us; skew 500 ppm
Sep 26 16:34:44 dist-test-slave-2zvq krb5kdc[24631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440484, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:34:44.673352 24644 init.cc:377] Logged in from keytab as kudu/127.11.8.254@KRBTEST.COM (short username kudu)
I20260926 16:34:44.674199 24644 webserver.cc:460] Webserver started at http://127.11.8.254:45009/ using document root <none> and password file <none>
I20260926 16:34:44.674755 24644 fs_manager.cc:359] Metadata directory not provided
I20260926 16:34:44.674814 24644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:34:44.675034 24644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:34:44.676944 24644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "4b1ba5b9a7b24550acc4c57bd68e1298"
format_stamp: "Formatted at 2026-09-26 16:34:44 on dist-test-slave-2zvq"
server_key: "2f79a74047f463e6a6920afa42e6e7e7"
server_key_iv: "554982c9e1a9a38445c37aedcb851199"
server_key_version: "encryptionkey@0"
I20260926 16:34:44.677495 24644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "4b1ba5b9a7b24550acc4c57bd68e1298"
format_stamp: "Formatted at 2026-09-26 16:34:44 on dist-test-slave-2zvq"
server_key: "2f79a74047f463e6a6920afa42e6e7e7"
server_key_iv: "554982c9e1a9a38445c37aedcb851199"
server_key_version: "encryptionkey@0"
I20260926 16:34:44.681037 24644 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260926 16:34:44.683480 24659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:44.684655 24644 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 16:34:44.684818 24644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "4b1ba5b9a7b24550acc4c57bd68e1298"
format_stamp: "Formatted at 2026-09-26 16:34:44 on dist-test-slave-2zvq"
server_key: "2f79a74047f463e6a6920afa42e6e7e7"
server_key_iv: "554982c9e1a9a38445c37aedcb851199"
server_key_version: "encryptionkey@0"
I20260926 16:34:44.684929 24644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:34:44.702485 24644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:34:44.705578 24644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:34:44.705770 24644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:34:44.712659 24644 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:34705
I20260926 16:34:44.712687 24711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:34705 every 8 connection(s)
I20260926 16:34:44.713536 24644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:34:44.716300 24712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:34:44.719859 24712 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.254" port: 34705 } has no permanent_uuid. Determining permanent_uuid...
I20260926 16:34:44.723121 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 24644
I20260926 16:34:44.723271 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:34:44.723537 11299 external_mini_cluster.cc:1428] Setting key 05538d6a6dde49cc8cb820d068cccdcd
Sep 26 16:34:44 dist-test-slave-2zvq krb5kdc[24631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440484, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.8.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.253@KRBTEST.COM" created.
I20260926 16:34:44.735576 24714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:44.721965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51568 (local address 127.11.8.254:34705)
0926 16:34:44.722422 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:44.722438 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:44.722658 (+ 220us) server_negotiation.cc:407] Connection header received
0926 16:34:44.723745 (+ 1087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:44.723750 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:44.723916 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:44.724102 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:34:44.724881 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:44.725728 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:44.727290 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:44.727742 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:44.727871 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:44.730656 (+ 2785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:34:44.730667 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:34:44.730676 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:34:44.730696 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:34:44.732364 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:44.733005 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:44.733009 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:44.733011 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:44.733142 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:44.733566 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:44.733569 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:44.733570 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:44.733928 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:34:44.734070 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:44.734508 (+ 438us) server_negotiation.cc:299] Negotiation successful
0926 16:34:44.734801 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":109,"server-negotiator.queue_time_us":330,"thread_start_us":175,"threads_started":1}
I20260926 16:34:44.735577 24715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:44.721965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:34705 (local address 127.0.0.1:51568)
0926 16:34:44.722484 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:44.722532 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 16:34:44.723443 (+ 911us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:34:44.724092 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:34:44.724097 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:34:44.724502 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:34:44.724738 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:44.724751 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:44.725862 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:44.725868 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:44.727155 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:44.727170 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:44.727913 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:44.727920 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:44.728082 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:44.728707 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:34:44.728734 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:34:44.730501 (+ 1767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:34:44.732545 (+ 2044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:44.732601 (+ 56us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:44.732614 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:44.732876 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:44.733234 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:44.733237 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:44.733238 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:44.733425 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:44.734045 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:34:44.734051 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:34:44.734184 (+ 133us) client_negotiation.cc:769] Sending connection context
0926 16:34:44.734386 (+ 202us) client_negotiation.cc:240] Negotiation successful
0926 16:34:44.734730 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"spinlock_wait_cycles":46336,"thread_start_us":175,"threads_started":1}
I20260926 16:34:44.737907 24712 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.253" port: 46735 } has no permanent_uuid. Determining permanent_uuid...
I20260926 16:34:44.738755 24715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:44.738428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.253:46735 (local address 127.0.0.1:54134)
0926 16:34:44.738576 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:44.738665 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.253:46735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260926 16:34:44.738978 24663 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.8.253:46735: connect: Connection refused (error 111)
I20260926 16:34:44.739914 24715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:44.739630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.253:46735 (local address 127.0.0.1:54148)
0926 16:34:44.739766 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:44.739830 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.253:46735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260926 16:34:44.740085 24712 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.253:46735: Network error: Client connection negotiation failed: client connection to 127.11.8.253:46735: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.253.keytab.
Entry for principal kudu/127.11.8.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.253.keytab.
I20260926 16:34:44.749876 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.253:46735
--webserver_interface=127.11.8.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.253.keytab
--principal=kudu/127.11.8.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:43851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:34705,127.11.8.253:46735,127.11.8.252:45237
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 16:34:44.789340 24712 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.253" port: 46735 } attempt: 1
I20260926 16:34:44.790441 24715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:44.789998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.253:46735 (local address 127.0.0.1:54154)
0926 16:34:44.790202 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:44.790325 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.253:46735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260926 16:34:44.791294 24715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:44.791014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.253:46735 (local address 127.0.0.1:54166)
0926 16:34:44.791146 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:44.791215 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.253:46735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260926 16:34:44.791435 24712 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.253:46735: Network error: Client connection negotiation failed: client connection to 127.11.8.253:46735: connect: Connection refused (error 111)
W20260926 16:34:44.857131 24719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:34:44.857371 24719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:34:44.857409 24719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260926 16:34:44.859614 24712 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.253" port: 46735 } attempt: 2
I20260926 16:34:44.860455 24715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:44.860140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.253:46735 (local address 127.0.0.1:54180)
0926 16:34:44.860301 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:44.860370 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.253:46735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260926 16:34:44.861169 24715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:44.860969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.253:46735 (local address 127.0.0.1:54186)
0926 16:34:44.861067 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:44.861114 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.253:46735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260926 16:34:44.861320 24712 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.253:46735: Network error: Client connection negotiation failed: client connection to 127.11.8.253:46735: connect: Connection refused (error 111)
W20260926 16:34:44.861862 24719 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:34:44.861930 24719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:34:44.861974 24719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:34:44.862010 24719 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:34:44.862043 24719 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:34:44.867259 24719 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:43851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:34705,127.11.8.253:46735,127.11.8.252:45237
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.253.keytab
--principal=kudu/127.11.8.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.253:46735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.8.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:34:44.868255 24719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:34:44.869223 24719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 16:34:44.877578 24719 server_base.cc:1034] running on GCE node
W20260926 16:34:44.877512 24724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:34:44.877511 24725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:34:44.877512 24727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:34:44.878325 24719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:34:44.879374 24719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:34:44.880537 24719 hybrid_clock.cc:648] HybridClock initialized: now 1790440484880526 us; error 27 us; skew 500 ppm
Sep 26 16:34:44 dist-test-slave-2zvq krb5kdc[24631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440484, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:34:44.883298 24719 init.cc:377] Logged in from keytab as kudu/127.11.8.253@KRBTEST.COM (short username kudu)
I20260926 16:34:44.884171 24719 webserver.cc:460] Webserver started at http://127.11.8.253:44639/ using document root <none> and password file <none>
I20260926 16:34:44.884703 24719 fs_manager.cc:359] Metadata directory not provided
I20260926 16:34:44.884809 24719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:34:44.885032 24719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:34:44.887071 24719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-1/data/instance:
uuid: "0e865cbba3ab4ef6a059856c5ff6809c"
format_stamp: "Formatted at 2026-09-26 16:34:44 on dist-test-slave-2zvq"
server_key: "3a51f2fa63a442ca2565242d72000575"
server_key_iv: "820fc4f0ffc3326d61c62cf6d1b20e7a"
server_key_version: "encryptionkey@0"
I20260926 16:34:44.887612 24719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-1/wal/instance:
uuid: "0e865cbba3ab4ef6a059856c5ff6809c"
format_stamp: "Formatted at 2026-09-26 16:34:44 on dist-test-slave-2zvq"
server_key: "3a51f2fa63a442ca2565242d72000575"
server_key_iv: "820fc4f0ffc3326d61c62cf6d1b20e7a"
server_key_version: "encryptionkey@0"
I20260926 16:34:44.891072 24719 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 16:34:44.893328 24734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:44.894529 24719 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:34:44.894654 24719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-1/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-1/wal
uuid: "0e865cbba3ab4ef6a059856c5ff6809c"
format_stamp: "Formatted at 2026-09-26 16:34:44 on dist-test-slave-2zvq"
server_key: "3a51f2fa63a442ca2565242d72000575"
server_key_iv: "820fc4f0ffc3326d61c62cf6d1b20e7a"
server_key_version: "encryptionkey@0"
I20260926 16:34:44.894764 24719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:34:44.923749 24719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:34:44.926584 24719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:34:44.926789 24719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:34:44.933609 24719 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.253:46735
I20260926 16:34:44.933626 24786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.253:46735 every 8 connection(s)
I20260926 16:34:44.934530 24719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-1/data/info.pb
I20260926 16:34:44.936131 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 24719
I20260926 16:34:44.936261 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-1/wal/instance
I20260926 16:34:44.936568 11299 external_mini_cluster.cc:1428] Setting key 107bd8d0498e68e00f4f0e07582a2f5f
I20260926 16:34:44.937711 24787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:34:44.941792 24787 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.254" port: 34705 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.11.8.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.252@KRBTEST.COM" created.
Sep 26 16:34:44 dist-test-slave-2zvq krb5kdc[24631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440484, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.253@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:34:44.952512 24712 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.253" port: 46735 } attempt: 3
I20260926 16:34:44.956038 24714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:44.943694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51572 (local address 127.11.8.254:34705)
0926 16:34:44.943852 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:44.943857 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:44.944660 (+ 803us) server_negotiation.cc:407] Connection header received
0926 16:34:44.945531 (+ 871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:44.945534 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:44.945611 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:44.945724 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:34:44.947289 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:44.947973 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:44.949118 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:44.949365 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:44.949477 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:44.951949 (+ 2472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:34:44.951957 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:34:44.951959 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:34:44.951983 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:34:44.953878 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:44.954456 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:44.954460 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:44.954461 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:44.954547 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:44.954871 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:44.954874 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:44.954876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:44.955087 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:34:44.955177 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:44.955755 (+ 578us) server_negotiation.cc:299] Negotiation successful
0926 16:34:44.955874 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 16:34:44.956882 24790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:44.944078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:34705 (local address 127.0.0.1:51572)
0926 16:34:44.944510 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:44.944551 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 16:34:44.945347 (+ 796us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:34:44.945923 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:34:44.945930 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:34:44.946399 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:34:44.947100 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:44.947122 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:44.948120 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:44.948123 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:44.948976 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:44.948985 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:44.949515 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:44.949519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:44.949738 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:44.950381 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:34:44.950397 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:34:44.951783 (+ 1386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:34:44.953999 (+ 2216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:44.954005 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:44.954025 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:44.954345 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:44.954639 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:44.954641 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:44.954643 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:44.954772 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:44.955185 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:34:44.955190 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:34:44.955485 (+ 295us) client_negotiation.cc:769] Sending connection context
0926 16:34:44.955669 (+ 184us) client_negotiation.cc:240] Negotiation successful
0926 16:34:44.955975 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":101,"threads_started":1}
I20260926 16:34:44.958642 24787 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.253" port: 46735 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 16:34:44 dist-test-slave-2zvq krb5kdc[24631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440484, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for kudu/127.11.8.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.252.keytab.
Entry for principal kudu/127.11.8.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.252.keytab.
I20260926 16:34:44.961853 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.252:45237
--webserver_interface=127.11.8.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.252.keytab
--principal=kudu/127.11.8.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:43851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:34705,127.11.8.253:46735,127.11.8.252:45237
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 16:34:44.963795 24715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:44.952988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.253:46735 (local address 127.0.0.1:54198)
0926 16:34:44.953161 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:44.953176 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:34:44.953257 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:34:44.953905 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:34:44.953908 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:34:44.954076 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:34:44.954171 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:44.954176 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:44.955437 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:44.955442 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:44.956118 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:44.956132 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:44.956809 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:44.956813 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:44.956927 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:44.957442 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:34:44.957457 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:34:44.959564 (+ 2107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:34:44.961825 (+ 2261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:44.961828 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:44.961830 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:44.962087 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:44.962450 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:44.962453 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:44.962455 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:44.962536 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:44.963203 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:34:44.963208 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:34:44.963318 (+ 110us) client_negotiation.cc:769] Sending connection context
0926 16:34:44.963428 (+ 110us) client_negotiation.cc:240] Negotiation successful
0926 16:34:44.963577 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91}
Sep 26 16:34:44 dist-test-slave-2zvq krb5kdc[24631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440484, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.253@KRBTEST.COM for kudu/127.11.8.253@KRBTEST.COM
I20260926 16:34:44.964506 24792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:44.953106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54198 (local address 127.11.8.253:46735)
0926 16:34:44.953465 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:44.953479 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:44.953506 (+ 27us) server_negotiation.cc:407] Connection header received
0926 16:34:44.953554 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:44.953561 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:44.953762 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:44.953924 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:34:44.954302 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:44.955327 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:44.956261 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:44.956689 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:44.956809 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:44.959695 (+ 2886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:34:44.959705 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:34:44.959732 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:34:44.959759 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:34:44.961696 (+ 1937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:44.962207 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:44.962212 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:44.962223 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:44.962344 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:44.962640 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:44.962643 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:44.962644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:44.962976 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:34:44.963143 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:44.964119 (+ 976us) server_negotiation.cc:299] Negotiation successful
0926 16:34:44.964303 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":158,"threads_started":1}
I20260926 16:34:44.966351 24712 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.252" port: 45237 } has no permanent_uuid. Determining permanent_uuid...
I20260926 16:34:44.967432 24715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:44.967141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:45237 (local address 127.0.0.1:46666)
0926 16:34:44.967285 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:44.967350 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.252:45237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260926 16:34:44.968201 24715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:44.967973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:45237 (local address 127.0.0.1:46682)
0926 16:34:44.968113 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:44.968154 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.252:45237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260926 16:34:44.968354 24712 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.252:45237: Network error: Client connection negotiation failed: client connection to 127.11.8.252:45237: connect: Connection refused (error 111)
I20260926 16:34:44.969416 24793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:44.959248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54202 (local address 127.11.8.253:46735)
0926 16:34:44.959532 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:44.959541 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:44.959554 (+ 13us) server_negotiation.cc:407] Connection header received
0926 16:34:44.959584 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:44.959590 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:44.959638 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:44.959715 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:34:44.960190 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:44.960883 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:44.961939 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:44.962198 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:44.962331 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:44.964986 (+ 2655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:34:44.964993 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:34:44.964995 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:34:44.965018 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:34:44.967377 (+ 2359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:44.968138 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:44.968143 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:44.968146 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:44.968237 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:44.968540 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:44.968543 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:44.968544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:44.968695 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:34:44.968777 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:44.969086 (+ 309us) server_negotiation.cc:299] Negotiation successful
0926 16:34:44.969285 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":82,"server-negotiator.queue_time_us":213,"thread_start_us":130,"threads_started":1}
I20260926 16:34:44.969467 24790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:44.959153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.253:46735 (local address 127.0.0.1:54202)
0926 16:34:44.959264 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:44.959278 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:34:44.959356 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:34:44.959725 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:34:44.959728 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:34:44.959928 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:34:44.960066 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:44.960076 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:44.960987 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:44.960992 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:44.961832 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:44.961839 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:44.962396 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:44.962403 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:44.962540 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:44.962984 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:34:44.963004 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:34:44.964352 (+ 1348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:34:44.967479 (+ 3127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:44.967485 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:44.967488 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:44.967745 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:44.968349 (+ 604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:44.968355 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:44.968358 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:44.968451 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:44.968802 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:34:44.968808 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:34:44.968919 (+ 111us) client_negotiation.cc:769] Sending connection context
0926 16:34:44.969043 (+ 124us) client_negotiation.cc:240] Negotiation successful
0926 16:34:44.969223 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":3}
I20260926 16:34:44.970059 24787 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.252" port: 45237 } has no permanent_uuid. Determining permanent_uuid...
I20260926 16:34:44.970767 24790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:44.970468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:45237 (local address 127.0.0.1:46698)
0926 16:34:44.970592 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:44.970679 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.252:45237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260926 16:34:44.970993 24735 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.8.252:45237: connect: Connection refused (error 111)
I20260926 16:34:44.971854 24790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:44.971585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:45237 (local address 127.0.0.1:46702)
0926 16:34:44.971713 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:44.971781 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.252:45237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260926 16:34:44.972024 24787 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.252:45237: Network error: Client connection negotiation failed: client connection to 127.11.8.252:45237: connect: Connection refused (error 111)
I20260926 16:34:44.999497 24712 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.252" port: 45237 } attempt: 1
I20260926 16:34:45.000279 24715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:44.999992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:45237 (local address 127.0.0.1:46704)
0926 16:34:45.000133 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:45.000197 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.252:45237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260926 16:34:45.001003 24715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:45.000795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:45237 (local address 127.0.0.1:46710)
0926 16:34:45.000898 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:45.000945 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.252:45237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260926 16:34:45.001153 24712 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.252:45237: Network error: Client connection negotiation failed: client connection to 127.11.8.252:45237: connect: Connection refused (error 111)
I20260926 16:34:45.021340 24787 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.252" port: 45237 } attempt: 1
I20260926 16:34:45.022346 24790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:45.021965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:45237 (local address 127.0.0.1:46718)
0926 16:34:45.022151 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:45.022227 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.252:45237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260926 16:34:45.023113 24790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:45.022863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:45237 (local address 127.0.0.1:46720)
0926 16:34:45.022976 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:45.023036 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.252:45237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260926 16:34:45.023262 24787 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.252:45237: Network error: Client connection negotiation failed: client connection to 127.11.8.252:45237: connect: Connection refused (error 111)
W20260926 16:34:45.073318 24794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:34:45.073607 24794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:34:45.073683 24794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260926 16:34:45.076309 24712 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.252" port: 45237 } attempt: 2
I20260926 16:34:45.077064 24715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:45.076782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:45237 (local address 127.0.0.1:46722)
0926 16:34:45.076924 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:45.076988 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.252:45237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260926 16:34:45.077823 24715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:45.077629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:45237 (local address 127.0.0.1:46738)
0926 16:34:45.077730 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:45.077774 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.252:45237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260926 16:34:45.077947 24712 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.252:45237: Network error: Client connection negotiation failed: client connection to 127.11.8.252:45237: connect: Connection refused (error 111)
W20260926 16:34:45.078120 24794 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:34:45.078197 24794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:34:45.078241 24794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:34:45.078297 24794 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:34:45.078333 24794 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:34:45.083515 24794 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:43851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:34705,127.11.8.253:46735,127.11.8.252:45237
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.252.keytab
--principal=kudu/127.11.8.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.252:45237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.8.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:34:45.084754 24794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:34:45.085768 24794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 16:34:45.091434 24787 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.252" port: 45237 } attempt: 2
I20260926 16:34:45.092231 24790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:45.091905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:45237 (local address 127.0.0.1:46744)
0926 16:34:45.092073 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:45.092140 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.252:45237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260926 16:34:45.093047 24790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:45.092796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:45237 (local address 127.0.0.1:46752)
0926 16:34:45.092922 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:45.092979 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.252:45237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260926 16:34:45.093304 24787 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.252:45237: Network error: Client connection negotiation failed: client connection to 127.11.8.252:45237: connect: Connection refused (error 111)
W20260926 16:34:45.094730 24800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:34:45.094748 24801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:34:45.094738 24803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:34:45.095129 24794 server_base.cc:1034] running on GCE node
I20260926 16:34:45.095748 24794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:34:45.096899 24794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:34:45.098088 24794 hybrid_clock.cc:648] HybridClock initialized: now 1790440485098072 us; error 25 us; skew 500 ppm
Sep 26 16:34:45 dist-test-slave-2zvq krb5kdc[24631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440485, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:34:45.101060 24794 init.cc:377] Logged in from keytab as kudu/127.11.8.252@KRBTEST.COM (short username kudu)
I20260926 16:34:45.102025 24794 webserver.cc:460] Webserver started at http://127.11.8.252:40263/ using document root <none> and password file <none>
I20260926 16:34:45.102653 24794 fs_manager.cc:359] Metadata directory not provided
I20260926 16:34:45.102744 24794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:34:45.102944 24794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:34:45.104982 24794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-2/data/instance:
uuid: "917bdd4fac90450cad194fb6abc767eb"
format_stamp: "Formatted at 2026-09-26 16:34:45 on dist-test-slave-2zvq"
server_key: "217fe0ad5c0842bf5e3f158e8f8fe496"
server_key_iv: "4b7e2b9f7a4133a4fce6a7e73aba08cf"
server_key_version: "encryptionkey@0"
I20260926 16:34:45.105549 24794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-2/wal/instance:
uuid: "917bdd4fac90450cad194fb6abc767eb"
format_stamp: "Formatted at 2026-09-26 16:34:45 on dist-test-slave-2zvq"
server_key: "217fe0ad5c0842bf5e3f158e8f8fe496"
server_key_iv: "4b7e2b9f7a4133a4fce6a7e73aba08cf"
server_key_version: "encryptionkey@0"
I20260926 16:34:45.109211 24794 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260926 16:34:45.111577 24810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:45.112725 24794 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 16:34:45.112871 24794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-2/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-2/wal
uuid: "917bdd4fac90450cad194fb6abc767eb"
format_stamp: "Formatted at 2026-09-26 16:34:45 on dist-test-slave-2zvq"
server_key: "217fe0ad5c0842bf5e3f158e8f8fe496"
server_key_iv: "4b7e2b9f7a4133a4fce6a7e73aba08cf"
server_key_version: "encryptionkey@0"
I20260926 16:34:45.112979 24794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:34:45.145262 24794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:34:45.148733 24794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:34:45.148940 24794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:34:45.156292 24794 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.252:45237
I20260926 16:34:45.156320 24862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.252:45237 every 8 connection(s)
I20260926 16:34:45.157217 24794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-2/data/info.pb
I20260926 16:34:45.158633 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 24794
I20260926 16:34:45.158762 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790440324823152-11299-0/minicluster-data/master-2/wal/instance
I20260926 16:34:45.159072 11299 external_mini_cluster.cc:1428] Setting key 0b55ca877622689574153fa4a5a5cebc
I20260926 16:34:45.160298 24863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:34:45.163944 24863 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.254" port: 34705 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 16:34:45 dist-test-slave-2zvq krb5kdc[24631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.252@KRBTEST.COM
Sep 26 16:34:45 dist-test-slave-2zvq krb5kdc[24631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440485, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.252@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:34:45.173250 24866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:45.160389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46762 (local address 127.11.8.252:45237)
0926 16:34:45.160815 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:45.160830 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:45.160865 (+ 35us) server_negotiation.cc:407] Connection header received
0926 16:34:45.161478 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:45.161494 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:45.161814 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:45.162138 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:34:45.162958 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:45.163938 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:45.164846 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:45.165316 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:45.165489 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:45.168156 (+ 2667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:34:45.168161 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:34:45.168169 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:34:45.168194 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:34:45.170015 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:45.170510 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:45.170522 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:45.170526 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:45.170630 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:45.170933 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:45.170937 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:45.170939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:45.171398 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:34:45.171615 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:45.171990 (+ 375us) server_negotiation.cc:299] Negotiation successful
0926 16:34:45.172291 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":141,"threads_started":1}
I20260926 16:34:45.174686 24867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:45.165500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:34705 (local address 127.0.0.1:51578)
0926 16:34:45.165898 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:45.165943 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 16:34:45.166061 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:34:45.166408 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:34:45.166412 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:34:45.166715 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:34:45.166836 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:45.166841 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:45.167691 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:45.167694 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:45.168316 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:45.168324 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:45.168921 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:45.168923 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:45.169099 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:45.169619 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:34:45.169631 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:34:45.171061 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:34:45.173032 (+ 1971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:45.173040 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:45.173043 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:45.173258 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:45.173590 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:45.173594 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:45.173596 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:45.173694 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:45.174138 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:34:45.174143 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:34:45.174287 (+ 144us) client_negotiation.cc:769] Sending connection context
0926 16:34:45.174433 (+ 146us) client_negotiation.cc:240] Negotiation successful
0926 16:34:45.174563 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":100,"threads_started":1}
I20260926 16:34:45.174775 24714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:45.165540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51578 (local address 127.11.8.254:34705)
0926 16:34:45.165681 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:45.165686 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:45.166038 (+ 352us) server_negotiation.cc:407] Connection header received
0926 16:34:45.166191 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:45.166195 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:45.166257 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:45.166375 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:34:45.166968 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:45.167524 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:45.168441 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:45.168775 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:45.168900 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:45.171207 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:34:45.171211 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:34:45.171213 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:34:45.171239 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:34:45.172911 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:45.173391 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:45.173394 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:45.173396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:45.173485 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:45.173815 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:45.173819 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:45.173820 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:45.174026 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:34:45.174128 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:45.174513 (+ 385us) server_negotiation.cc:299] Negotiation successful
0926 16:34:45.174636 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 16:34:45.176347 24863 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.253" port: 46735 } has no permanent_uuid. Determining permanent_uuid...
I20260926 16:34:45.177109 24712 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.252" port: 45237 } attempt: 3
Sep 26 16:34:45 dist-test-slave-2zvq krb5kdc[24631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440485, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.252@KRBTEST.COM for kudu/127.11.8.253@KRBTEST.COM
Sep 26 16:34:45 dist-test-slave-2zvq krb5kdc[24631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
Sep 26 16:34:45 dist-test-slave-2zvq krb5kdc[24631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440484, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for kudu/127.11.8.252@KRBTEST.COM
I20260926 16:34:45.184518 24787 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.252" port: 45237 } attempt: 3
I20260926 16:34:45.186677 24715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:45.177458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:45237 (local address 127.0.0.1:46776)
0926 16:34:45.177677 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:45.177692 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:34:45.177757 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:34:45.177987 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:34:45.177990 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:34:45.178174 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:34:45.178311 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:45.178318 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:45.179336 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:45.179340 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:45.180132 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:45.180143 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:45.180735 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:45.180741 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:45.180847 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:45.181323 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:34:45.181336 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:34:45.182966 (+ 1630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:34:45.184971 (+ 2005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:45.184977 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:45.184982 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:45.185243 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:45.185573 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:45.185576 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:45.185578 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:45.185691 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:45.186130 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:34:45.186135 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:34:45.186231 (+ 96us) client_negotiation.cc:769] Sending connection context
0926 16:34:45.186362 (+ 131us) client_negotiation.cc:240] Negotiation successful
0926 16:34:45.186500 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144}
I20260926 16:34:45.186997 24714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:45.175855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51580 (local address 127.11.8.254:34705)
0926 16:34:45.175963 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:45.175969 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:45.175987 (+ 18us) server_negotiation.cc:407] Connection header received
0926 16:34:45.176024 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:45.176027 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:45.176062 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:45.176162 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:34:45.176876 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:45.177623 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:45.178811 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:45.179023 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:45.179167 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:45.182827 (+ 3660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:34:45.182830 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:34:45.182832 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:34:45.182853 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:34:45.184479 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:45.185115 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:45.185119 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:45.185121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:45.185216 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:45.185549 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:45.185554 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:45.185556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:45.185794 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:34:45.186020 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:45.186706 (+ 686us) server_negotiation.cc:299] Negotiation successful
0926 16:34:45.186829 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 16:34:45.187654 24793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:45.176955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54204 (local address 127.11.8.253:46735)
0926 16:34:45.177061 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:45.177065 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:45.177081 (+ 16us) server_negotiation.cc:407] Connection header received
0926 16:34:45.177200 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:45.177203 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:45.177251 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:45.177333 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:34:45.177829 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:45.178530 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:45.179577 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:45.179948 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:45.180051 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:45.183948 (+ 3897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:34:45.183955 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:34:45.183959 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:34:45.183992 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:34:45.185861 (+ 1869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:45.186322 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:45.186325 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:45.186327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:45.186422 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:45.186734 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:45.186737 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:45.186740 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:45.186972 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:34:45.187046 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:45.187374 (+ 328us) server_negotiation.cc:299] Negotiation successful
0926 16:34:45.187497 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 16:34:45.187654 24866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:45.177680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46776 (local address 127.11.8.252:45237)
0926 16:34:45.177786 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:45.177792 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:45.177807 (+ 15us) server_negotiation.cc:407] Connection header received
0926 16:34:45.177835 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:45.177838 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:45.177880 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:45.177952 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:34:45.178424 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:45.179189 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:45.180251 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:45.180578 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:45.180714 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:45.183111 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:34:45.183116 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:34:45.183132 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:34:45.183155 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:34:45.184829 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:45.185368 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:45.185374 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:45.185378 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:45.185461 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:45.185787 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:45.185792 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:45.185793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:45.186001 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:34:45.187358 (+ 1357us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:45.187432 (+ 74us) server_negotiation.cc:299] Negotiation successful
0926 16:34:45.187530 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 16:34:45.188423 24867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:45.176834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.253:46735 (local address 127.0.0.1:54204)
0926 16:34:45.176963 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:45.176977 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:34:45.177048 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:34:45.177444 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:34:45.177446 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:34:45.177568 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:34:45.177678 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:45.177685 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:45.178661 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:45.178663 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:45.179462 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:45.179474 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:45.180058 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:45.180061 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:45.180129 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:45.180604 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:34:45.180621 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:34:45.183340 (+ 2719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:34:45.185953 (+ 2613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:45.185956 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:45.185958 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:45.186193 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:45.186536 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:45.186539 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:45.186542 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:45.186629 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:45.187120 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:34:45.187123 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:34:45.187202 (+ 79us) client_negotiation.cc:769] Sending connection context
0926 16:34:45.188137 (+ 935us) client_negotiation.cc:240] Negotiation successful
0926 16:34:45.188271 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"spinlock_wait_cycles":3968}
I20260926 16:34:45.190987 24863 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.252" port: 45237 } has no permanent_uuid. Determining permanent_uuid...
I20260926 16:34:45.190990 24712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b1ba5b9a7b24550acc4c57bd68e1298: Bootstrap starting.
I20260926 16:34:45.193605 24712 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b1ba5b9a7b24550acc4c57bd68e1298: Neither blocks nor log segments found. Creating new log.
I20260926 16:34:45.194440 24712 log.cc:824] T 00000000000000000000000000000000 P 4b1ba5b9a7b24550acc4c57bd68e1298: Log is configured to *not* fsync() on all Append() calls
Sep 26 16:34:45 dist-test-slave-2zvq krb5kdc[24631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440484, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.253@KRBTEST.COM for kudu/127.11.8.252@KRBTEST.COM
Sep 26 16:34:45 dist-test-slave-2zvq krb5kdc[24631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.253@KRBTEST.COM
Sep 26 16:34:45 dist-test-slave-2zvq krb5kdc[24631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440485, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.252@KRBTEST.COM for kudu/127.11.8.252@KRBTEST.COM
I20260926 16:34:45.197916 24712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b1ba5b9a7b24550acc4c57bd68e1298: No bootstrap required, opened a new log
I20260926 16:34:45.200543 24790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:45.185153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:45237 (local address 127.0.0.1:46792)
0926 16:34:45.185334 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:45.185349 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:34:45.185414 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:34:45.186391 (+ 977us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:34:45.186394 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:34:45.186561 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:34:45.186673 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:45.186680 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:45.188315 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:45.188318 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:45.189076 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:45.189109 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:45.189758 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:45.189765 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:45.189863 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:45.190512 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:34:45.190525 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:34:45.195772 (+ 5247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:34:45.198497 (+ 2725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:45.198501 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:45.198504 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:45.198783 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:45.199175 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:45.199179 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:45.199182 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:45.199276 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:45.200062 (+ 786us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:34:45.200066 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:34:45.200160 (+ 94us) client_negotiation.cc:769] Sending connection context
0926 16:34:45.200266 (+ 106us) client_negotiation.cc:240] Negotiation successful
0926 16:34:45.200387 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106}
I20260926 16:34:45.200943 24866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:45.191498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46794 (local address 127.11.8.252:45237)
0926 16:34:45.191585 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:45.191589 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:45.191602 (+ 13us) server_negotiation.cc:407] Connection header received
0926 16:34:45.191738 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:45.191744 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:45.191793 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:45.191870 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:34:45.192250 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:45.192708 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:45.193753 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:45.194144 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:45.194244 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:45.196994 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:34:45.196998 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:34:45.197002 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:34:45.197028 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:34:45.198794 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:45.199677 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:45.199683 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:45.199685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:45.199762 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:45.200208 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:45.200210 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:45.200211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:45.200357 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:34:45.200462 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:45.200726 (+ 264us) server_negotiation.cc:299] Negotiation successful
0926 16:34:45.200824 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 16:34:45.201329 24867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:45.191421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:45237 (local address 127.0.0.1:46794)
0926 16:34:45.191553 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:45.191566 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:34:45.191639 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:34:45.191876 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:34:45.191879 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:34:45.192038 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:34:45.192118 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:45.192123 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:45.192806 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:45.192812 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:45.193613 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:45.193626 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:45.194300 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:45.194305 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:45.194386 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:45.194782 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:34:45.194801 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:34:45.196841 (+ 2040us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:34:45.198894 (+ 2053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:45.198896 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:45.198898 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:45.199584 (+ 686us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:45.199852 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:45.199854 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:45.199856 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:45.199933 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:45.200454 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:34:45.200456 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:34:45.200531 (+ 75us) client_negotiation.cc:769] Sending connection context
0926 16:34:45.201123 (+ 592us) client_negotiation.cc:240] Negotiation successful
0926 16:34:45.201233 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260926 16:34:45.201996 24868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:45.184963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46792 (local address 127.11.8.252:45237)
0926 16:34:45.186118 (+ 1155us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:45.186128 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:45.186139 (+ 11us) server_negotiation.cc:407] Connection header received
0926 16:34:45.186181 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:45.186184 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:45.186225 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:45.186785 (+ 560us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:34:45.186871 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:45.188193 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:45.189235 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:45.189628 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:45.189762 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:45.195920 (+ 6158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:34:45.195923 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:34:45.195924 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:34:45.195944 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:34:45.198371 (+ 2427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:45.198935 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:45.198938 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:45.198943 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:45.199060 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:45.199358 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:45.199362 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:45.199364 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:45.199609 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:34:45.199715 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:45.201735 (+ 2020us) server_negotiation.cc:299] Negotiation successful
0926 16:34:45.201837 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1089,"spinlock_wait_cycles":10624,"thread_start_us":109,"threads_started":1}
I20260926 16:34:45.202663 24712 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b1ba5b9a7b24550acc4c57bd68e1298 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b1ba5b9a7b24550acc4c57bd68e1298" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 34705 } } peers { permanent_uuid: "0e865cbba3ab4ef6a059856c5ff6809c" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 46735 } } peers { permanent_uuid: "917bdd4fac90450cad194fb6abc767eb" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 45237 } }
I20260926 16:34:45.202883 24712 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b1ba5b9a7b24550acc4c57bd68e1298 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:34:45.202931 24712 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b1ba5b9a7b24550acc4c57bd68e1298 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b1ba5b9a7b24550acc4c57bd68e1298, State: Initialized, Role: FOLLOWER
I20260926 16:34:45.203403 24712 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b1ba5b9a7b24550acc4c57bd68e1298 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b1ba5b9a7b24550acc4c57bd68e1298" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 34705 } } peers { permanent_uuid: "0e865cbba3ab4ef6a059856c5ff6809c" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 46735 } } peers { permanent_uuid: "917bdd4fac90450cad194fb6abc767eb" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 45237 } }
I20260926 16:34:45.203894 24870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b1ba5b9a7b24550acc4c57bd68e1298 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b1ba5b9a7b24550acc4c57bd68e1298" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 34705 } } peers { permanent_uuid: "0e865cbba3ab4ef6a059856c5ff6809c" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 46735 } } peers { permanent_uuid: "917bdd4fac90450cad194fb6abc767eb" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 45237 } } }
I20260926 16:34:45.204018 24870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b1ba5b9a7b24550acc4c57bd68e1298 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:34:45.204160 24863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 917bdd4fac90450cad194fb6abc767eb: Bootstrap starting.
I20260926 16:34:45.204715 24712 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b1ba5b9a7b24550acc4c57bd68e1298 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:34:45.204973 24787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e865cbba3ab4ef6a059856c5ff6809c: Bootstrap starting.
I20260926 16:34:45.206458 24863 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 917bdd4fac90450cad194fb6abc767eb: Neither blocks nor log segments found. Creating new log.
W20260926 16:34:45.207211 24665 tablet.cc:2369] T 00000000000000000000000000000000 P 4b1ba5b9a7b24550acc4c57bd68e1298: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:34:45.207309 24863 log.cc:824] T 00000000000000000000000000000000 P 917bdd4fac90450cad194fb6abc767eb: Log is configured to *not* fsync() on all Append() calls
I20260926 16:34:45.207497 24787 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e865cbba3ab4ef6a059856c5ff6809c: Neither blocks nor log segments found. Creating new log.
I20260926 16:34:45.208277 24787 log.cc:824] T 00000000000000000000000000000000 P 0e865cbba3ab4ef6a059856c5ff6809c: Log is configured to *not* fsync() on all Append() calls
I20260926 16:34:45.210060 24793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:45.188766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54214 (local address 127.11.8.253:46735)
0926 16:34:45.188851 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:45.188855 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:45.189038 (+ 183us) server_negotiation.cc:407] Connection header received
0926 16:34:45.189145 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:45.189151 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:45.189213 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:45.189259 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:34:45.190078 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:45.190793 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:45.191781 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:45.192111 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:45.192267 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:45.198870 (+ 6603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:34:45.198874 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:34:45.198876 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:34:45.198899 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:34:45.201444 (+ 2545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:45.202498 (+ 1054us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:45.202501 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:45.202503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:45.202583 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:45.208981 (+ 6398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:45.208984 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:45.208986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:45.209201 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:34:45.209350 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:45.209751 (+ 401us) server_negotiation.cc:299] Negotiation successful
0926 16:34:45.209884 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260926 16:34:45.210129 24881 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4b1ba5b9a7b24550acc4c57bd68e1298: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 16:34:45.210204 24881 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 16:34:45.211699 24863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 917bdd4fac90450cad194fb6abc767eb: No bootstrap required, opened a new log
I20260926 16:34:45.212044 24787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e865cbba3ab4ef6a059856c5ff6809c: No bootstrap required, opened a new log
I20260926 16:34:45.209584 24644 master_runner.cc:186] Error getting master registration for 127.11.8.253:46735: OK, instance_id { permanent_uuid: "0e865cbba3ab4ef6a059856c5ff6809c" instance_seqno: 1790440484931671 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 16:34:45.213511 24644 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 16:34:45.215116 24863 raft_consensus.cc:348] T 00000000000000000000000000000000 P 917bdd4fac90450cad194fb6abc767eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b1ba5b9a7b24550acc4c57bd68e1298" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 34705 } } peers { permanent_uuid: "0e865cbba3ab4ef6a059856c5ff6809c" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 46735 } } peers { permanent_uuid: "917bdd4fac90450cad194fb6abc767eb" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 45237 } }
I20260926 16:34:45.215426 24863 raft_consensus.cc:374] T 00000000000000000000000000000000 P 917bdd4fac90450cad194fb6abc767eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:34:45.215530 24863 raft_consensus.cc:729] T 00000000000000000000000000000000 P 917bdd4fac90450cad194fb6abc767eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 917bdd4fac90450cad194fb6abc767eb, State: Initialized, Role: FOLLOWER
I20260926 16:34:45.216089 24863 consensus_queue.cc:260] T 00000000000000000000000000000000 P 917bdd4fac90450cad194fb6abc767eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b1ba5b9a7b24550acc4c57bd68e1298" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 34705 } } peers { permanent_uuid: "0e865cbba3ab4ef6a059856c5ff6809c" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 46735 } } peers { permanent_uuid: "917bdd4fac90450cad194fb6abc767eb" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 45237 } }
I20260926 16:34:45.216248 24787 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e865cbba3ab4ef6a059856c5ff6809c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b1ba5b9a7b24550acc4c57bd68e1298" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 34705 } } peers { permanent_uuid: "0e865cbba3ab4ef6a059856c5ff6809c" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 46735 } } peers { permanent_uuid: "917bdd4fac90450cad194fb6abc767eb" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 45237 } }
I20260926 16:34:45.216446 24787 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e865cbba3ab4ef6a059856c5ff6809c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:34:45.216514 24787 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e865cbba3ab4ef6a059856c5ff6809c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e865cbba3ab4ef6a059856c5ff6809c, State: Initialized, Role: FOLLOWER
I20260926 16:34:45.216574 11299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:34:45.216948 24787 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e865cbba3ab4ef6a059856c5ff6809c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b1ba5b9a7b24550acc4c57bd68e1298" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 34705 } } peers { permanent_uuid: "0e865cbba3ab4ef6a059856c5ff6809c" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 46735 } } peers { permanent_uuid: "917bdd4fac90450cad194fb6abc767eb" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 45237 } }
I20260926 16:34:45.217852 24863 sys_catalog.cc:564] T 00000000000000000000000000000000 P 917bdd4fac90450cad194fb6abc767eb [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:34:45.218185 24787 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e865cbba3ab4ef6a059856c5ff6809c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:34:45.221330 24886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e865cbba3ab4ef6a059856c5ff6809c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b1ba5b9a7b24550acc4c57bd68e1298" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 34705 } } peers { permanent_uuid: "0e865cbba3ab4ef6a059856c5ff6809c" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 46735 } } peers { permanent_uuid: "917bdd4fac90450cad194fb6abc767eb" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 45237 } } }
I20260926 16:34:45.223266 24886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e865cbba3ab4ef6a059856c5ff6809c [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:34:45.227779 24794 master_runner.cc:263] No leader master found from master 917bdd4fac90450cad194fb6abc767eb
I20260926 16:34:45.227840 24794 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 16:34:45.227942 24719 master_runner.cc:263] No leader master found from master 0e865cbba3ab4ef6a059856c5ff6809c
I20260926 16:34:45.227983 24719 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 16:34:45.228641 24906 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 917bdd4fac90450cad194fb6abc767eb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 16:34:45.228708 24906 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260926 16:34:45.228891 24912 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0e865cbba3ab4ef6a059856c5ff6809c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 16:34:45.229001 24912 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 16:34:45.236804 24885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 917bdd4fac90450cad194fb6abc767eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b1ba5b9a7b24550acc4c57bd68e1298" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 34705 } } peers { permanent_uuid: "0e865cbba3ab4ef6a059856c5ff6809c" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 46735 } } peers { permanent_uuid: "917bdd4fac90450cad194fb6abc767eb" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 45237 } } }
I20260926 16:34:45.237092 24885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 917bdd4fac90450cad194fb6abc767eb [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:34:45.244886 24793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:45.222657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54220 (local address 127.11.8.253:46735)
0926 16:34:45.222766 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:45.222770 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:45.223784 (+ 1014us) server_negotiation.cc:407] Connection header received
0926 16:34:45.224465 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:45.224469 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:45.224512 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:45.224559 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:34:45.228852 (+ 4293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:45.229555 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:45.233609 (+ 4054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:45.233959 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:45.234034 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:45.239303 (+ 5269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:34:45.239307 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:34:45.239310 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:34:45.239335 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:34:45.241995 (+ 2660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:45.243535 (+ 1540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:45.243539 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:45.243541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:45.243640 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:45.243945 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:45.243949 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:45.243951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:45.244151 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:34:45.244272 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:45.244599 (+ 327us) server_negotiation.cc:299] Negotiation successful
0926 16:34:45.244731 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 16:34:45.251376 24868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:45.221802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46806 (local address 127.11.8.252:45237)
0926 16:34:45.221907 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:45.221911 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:45.224297 (+ 2386us) server_negotiation.cc:407] Connection header received
0926 16:34:45.226752 (+ 2455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:45.226755 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:45.226797 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:45.226848 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:34:45.231766 (+ 4918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:45.232504 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:45.235661 (+ 3157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:45.236045 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:45.236119 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:45.240277 (+ 4158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:34:45.240281 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:34:45.240284 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:34:45.240311 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:34:45.243033 (+ 2722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:45.249066 (+ 6033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:45.249070 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:45.249072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:45.249174 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:45.249835 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:45.249841 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:45.249843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:45.250040 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:34:45.250156 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:45.251095 (+ 939us) server_negotiation.cc:299] Negotiation successful
0926 16:34:45.251222 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 16:34:45.253177 24714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:45.222228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51588 (local address 127.11.8.254:34705)
0926 16:34:45.222361 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:45.222365 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:45.223996 (+ 1631us) server_negotiation.cc:407] Connection header received
0926 16:34:45.225583 (+ 1587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:45.225587 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:45.225645 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:45.225697 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:34:45.227824 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:45.228612 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:45.234569 (+ 5957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:45.234917 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:45.234994 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:45.246901 (+ 11907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:34:45.246905 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:34:45.246908 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:34:45.246938 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:34:45.248589 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:45.249465 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:45.249468 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:45.249470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:45.249545 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:45.250465 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:45.250468 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:45.250470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:45.250685 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:34:45.250787 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:45.252914 (+ 2127us) server_negotiation.cc:299] Negotiation successful
0926 16:34:45.253031 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 16:34:45.381877 24870 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b1ba5b9a7b24550acc4c57bd68e1298 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:34:45.382076 24870 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b1ba5b9a7b24550acc4c57bd68e1298 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b1ba5b9a7b24550acc4c57bd68e1298" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 34705 } } peers { permanent_uuid: "0e865cbba3ab4ef6a059856c5ff6809c" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 46735 } } peers { permanent_uuid: "917bdd4fac90450cad194fb6abc767eb" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 45237 } }
I20260926 16:34:45.383134 24870 leader_election.cc:288] T 00000000000000000000000000000000 P 4b1ba5b9a7b24550acc4c57bd68e1298 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e865cbba3ab4ef6a059856c5ff6809c (127.11.8.253:46735), 917bdd4fac90450cad194fb6abc767eb (127.11.8.252:45237)
I20260926 16:34:45.384011 24761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4b1ba5b9a7b24550acc4c57bd68e1298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e865cbba3ab4ef6a059856c5ff6809c" is_pre_election: true
I20260926 16:34:45.384321 24761 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0e865cbba3ab4ef6a059856c5ff6809c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b1ba5b9a7b24550acc4c57bd68e1298 in term 0.
I20260926 16:34:45.384888 24663 leader_election.cc:302] T 00000000000000000000000000000000 P 4b1ba5b9a7b24550acc4c57bd68e1298 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e865cbba3ab4ef6a059856c5ff6809c, 4b1ba5b9a7b24550acc4c57bd68e1298; no voters:
I20260926 16:34:45.385283 24870 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b1ba5b9a7b24550acc4c57bd68e1298 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:34:45.385367 24870 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b1ba5b9a7b24550acc4c57bd68e1298 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:34:45.385409 24870 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b1ba5b9a7b24550acc4c57bd68e1298 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:34:45.386698 24870 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b1ba5b9a7b24550acc4c57bd68e1298 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b1ba5b9a7b24550acc4c57bd68e1298" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 34705 } } peers { permanent_uuid: "0e865cbba3ab4ef6a059856c5ff6809c" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 46735 } } peers { permanent_uuid: "917bdd4fac90450cad194fb6abc767eb" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 45237 } }
I20260926 16:34:45.387040 24870 leader_election.cc:288] T 00000000000000000000000000000000 P 4b1ba5b9a7b24550acc4c57bd68e1298 [CANDIDATE]: Term 1 election: Requested vote from peers 0e865cbba3ab4ef6a059856c5ff6809c (127.11.8.253:46735), 917bdd4fac90450cad194fb6abc767eb (127.11.8.252:45237)
I20260926 16:34:45.387533 24761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4b1ba5b9a7b24550acc4c57bd68e1298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e865cbba3ab4ef6a059856c5ff6809c"
I20260926 16:34:45.387673 24761 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e865cbba3ab4ef6a059856c5ff6809c [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:34:45.387836 24837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4b1ba5b9a7b24550acc4c57bd68e1298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "917bdd4fac90450cad194fb6abc767eb" is_pre_election: true
I20260926 16:34:45.388145 24837 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 917bdd4fac90450cad194fb6abc767eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b1ba5b9a7b24550acc4c57bd68e1298 in term 0.
I20260926 16:34:45.388962 24761 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0e865cbba3ab4ef6a059856c5ff6809c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b1ba5b9a7b24550acc4c57bd68e1298 in term 1.
I20260926 16:34:45.389216 24836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4b1ba5b9a7b24550acc4c57bd68e1298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "917bdd4fac90450cad194fb6abc767eb"
I20260926 16:34:45.389312 24663 leader_election.cc:302] T 00000000000000000000000000000000 P 4b1ba5b9a7b24550acc4c57bd68e1298 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e865cbba3ab4ef6a059856c5ff6809c, 4b1ba5b9a7b24550acc4c57bd68e1298; no voters:
I20260926 16:34:45.389343 24836 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 917bdd4fac90450cad194fb6abc767eb [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:34:45.390363 24836 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 917bdd4fac90450cad194fb6abc767eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b1ba5b9a7b24550acc4c57bd68e1298 in term 1.
I20260926 16:34:45.391038 24870 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b1ba5b9a7b24550acc4c57bd68e1298 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:34:45.391330 24870 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b1ba5b9a7b24550acc4c57bd68e1298 [term 1 LEADER]: Becoming Leader. State: Replica: 4b1ba5b9a7b24550acc4c57bd68e1298, State: Running, Role: LEADER
I20260926 16:34:45.391685 24870 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b1ba5b9a7b24550acc4c57bd68e1298 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b1ba5b9a7b24550acc4c57bd68e1298" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 34705 } } peers { permanent_uuid: "0e865cbba3ab4ef6a059856c5ff6809c" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 46735 } } peers { permanent_uuid: "917bdd4fac90450cad194fb6abc767eb" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 45237 } }
I20260926 16:34:45.393800 24918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b1ba5b9a7b24550acc4c57bd68e1298 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b1ba5b9a7b24550acc4c57bd68e1298. Latest consensus state: current_term: 1 leader_uuid: "4b1ba5b9a7b24550acc4c57bd68e1298" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b1ba5b9a7b24550acc4c57bd68e1298" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 34705 } } peers { permanent_uuid: "0e865cbba3ab4ef6a059856c5ff6809c" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 46735 } } peers { permanent_uuid: "917bdd4fac90450cad194fb6abc767eb" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 45237 } } }
I20260926 16:34:45.393941 24918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b1ba5b9a7b24550acc4c57bd68e1298 [sys.catalog]: This master's current role is: LEADER
I20260926 16:34:45.395850 24920 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:34:45.396744 24920 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260926 16:34:45.400382 24816 tablet.cc:2369] T 00000000000000000000000000000000 P 917bdd4fac90450cad194fb6abc767eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:34:45.402005 24761 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0e865cbba3ab4ef6a059856c5ff6809c [term 1 FOLLOWER]: Refusing update from remote peer 4b1ba5b9a7b24550acc4c57bd68e1298: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:34:45.402046 24836 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 917bdd4fac90450cad194fb6abc767eb [term 1 FOLLOWER]: Refusing update from remote peer 4b1ba5b9a7b24550acc4c57bd68e1298: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:34:45.402679 24918 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4b1ba5b9a7b24550acc4c57bd68e1298 [LEADER]: Connected to new peer: Peer: permanent_uuid: "917bdd4fac90450cad194fb6abc767eb" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 45237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:34:45.403132 24918 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4b1ba5b9a7b24550acc4c57bd68e1298 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e865cbba3ab4ef6a059856c5ff6809c" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 46735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:34:45.406993 24885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 917bdd4fac90450cad194fb6abc767eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b1ba5b9a7b24550acc4c57bd68e1298. Latest consensus state: current_term: 1 leader_uuid: "4b1ba5b9a7b24550acc4c57bd68e1298" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b1ba5b9a7b24550acc4c57bd68e1298" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 34705 } } peers { permanent_uuid: "0e865cbba3ab4ef6a059856c5ff6809c" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 46735 } } peers { permanent_uuid: "917bdd4fac90450cad194fb6abc767eb" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 45237 } } }
I20260926 16:34:45.407128 24885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 917bdd4fac90450cad194fb6abc767eb [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:34:45.407814 24886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e865cbba3ab4ef6a059856c5ff6809c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b1ba5b9a7b24550acc4c57bd68e1298. Latest consensus state: current_term: 1 leader_uuid: "4b1ba5b9a7b24550acc4c57bd68e1298" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b1ba5b9a7b24550acc4c57bd68e1298" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 34705 } } peers { permanent_uuid: "0e865cbba3ab4ef6a059856c5ff6809c" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 46735 } } peers { permanent_uuid: "917bdd4fac90450cad194fb6abc767eb" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 45237 } } }
I20260926 16:34:45.407954 24886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e865cbba3ab4ef6a059856c5ff6809c [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:34:45.412858 24918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b1ba5b9a7b24550acc4c57bd68e1298 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4b1ba5b9a7b24550acc4c57bd68e1298" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b1ba5b9a7b24550acc4c57bd68e1298" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 34705 } } peers { permanent_uuid: "0e865cbba3ab4ef6a059856c5ff6809c" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 46735 } } peers { permanent_uuid: "917bdd4fac90450cad194fb6abc767eb" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 45237 } } }
I20260926 16:34:45.412994 24918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b1ba5b9a7b24550acc4c57bd68e1298 [sys.catalog]: This master's current role is: LEADER
I20260926 16:34:45.413250 24921 mvcc.cc:205] Tried to move back new op lower bound from 7333644228198199296 to 7333644228167483392. Current Snapshot: MvccSnapshot[applied={T|T < 7333644228198199296}]
I20260926 16:34:45.413367 24886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e865cbba3ab4ef6a059856c5ff6809c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4b1ba5b9a7b24550acc4c57bd68e1298" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b1ba5b9a7b24550acc4c57bd68e1298" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 34705 } } peers { permanent_uuid: "0e865cbba3ab4ef6a059856c5ff6809c" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 46735 } } peers { permanent_uuid: "917bdd4fac90450cad194fb6abc767eb" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 45237 } } }
I20260926 16:34:45.413460 24886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e865cbba3ab4ef6a059856c5ff6809c [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:34:45.415512 24918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b1ba5b9a7b24550acc4c57bd68e1298 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4b1ba5b9a7b24550acc4c57bd68e1298" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b1ba5b9a7b24550acc4c57bd68e1298" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 34705 } } peers { permanent_uuid: "0e865cbba3ab4ef6a059856c5ff6809c" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 46735 } } peers { permanent_uuid: "917bdd4fac90450cad194fb6abc767eb" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 45237 } } }
I20260926 16:34:45.415623 24918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b1ba5b9a7b24550acc4c57bd68e1298 [sys.catalog]: This master's current role is: LEADER
I20260926 16:34:45.417075 24885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 917bdd4fac90450cad194fb6abc767eb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4b1ba5b9a7b24550acc4c57bd68e1298" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b1ba5b9a7b24550acc4c57bd68e1298" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 34705 } } peers { permanent_uuid: "0e865cbba3ab4ef6a059856c5ff6809c" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 46735 } } peers { permanent_uuid: "917bdd4fac90450cad194fb6abc767eb" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 45237 } } }
I20260926 16:34:45.417352 24885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 917bdd4fac90450cad194fb6abc767eb [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:34:45.418043 24922 mvcc.cc:205] Tried to move back new op lower bound from 7333644228198199296 to 7333644228167483392. Current Snapshot: MvccSnapshot[applied={T|T < 7333644228198199296}]
I20260926 16:34:45.418290 24920 catalog_manager.cc:1347] Generated new cluster ID: efaa2b6b0a6546b6b60edd3edfb53a00
I20260926 16:34:45.418346 24920 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:34:45.420609 24924 mvcc.cc:205] Tried to move back new op lower bound from 7333644228198199296 to 7333644228167483392. Current Snapshot: MvccSnapshot[applied={T|T < 7333644228198199296 or (T in {7333644228198199296})}]
I20260926 16:34:45.431762 24920 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:34:45.432956 24920 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:34:45.440907 24920 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b1ba5b9a7b24550acc4c57bd68e1298: Generated new TSK 0
I20260926 16:34:45.441779 24920 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:34:46.230227 24912 catalog_manager.cc:1259] Loaded cluster ID: efaa2b6b0a6546b6b60edd3edfb53a00
I20260926 16:34:46.230195 24906 catalog_manager.cc:1259] Loaded cluster ID: efaa2b6b0a6546b6b60edd3edfb53a00
I20260926 16:34:46.230314 24912 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0e865cbba3ab4ef6a059856c5ff6809c: loading cluster ID for follower catalog manager: success
I20260926 16:34:46.230315 24906 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 917bdd4fac90450cad194fb6abc767eb: loading cluster ID for follower catalog manager: success
I20260926 16:34:46.232566 24912 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0e865cbba3ab4ef6a059856c5ff6809c: acquiring CA information for follower catalog manager: success
I20260926 16:34:46.232566 24906 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 917bdd4fac90450cad194fb6abc767eb: acquiring CA information for follower catalog manager: success
I20260926 16:34:46.234054 24912 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0e865cbba3ab4ef6a059856c5ff6809c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 16:34:46.234054 24906 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 917bdd4fac90450cad194fb6abc767eb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 16:34:46.496589 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 24644
I20260926 16:34:46.503016 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 24719
I20260926 16:34:46.508973 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 24794
2026-09-26T16:34:46Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3131 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:34:46 dist-test-slave-2zvq krb5kdc[24935](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:34:46 dist-test-slave-2zvq krb5kdc[24935](info): set up 2 sockets
Sep 26 16:34:46 dist-test-slave-2zvq krb5kdc[24935](info): commencing operation
krb5kdc: starting...
W20260926 16:34:47.552256 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:34:47 dist-test-slave-2zvq krb5kdc[24935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440487, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:34:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:34:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
I20260926 16:34:47.672780 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:33519
--webserver_interface=127.11.8.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:35209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:33519
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:34:47.781836 24948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:34:47.782146 24948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:34:47.782233 24948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:34:47.786218 24948 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:34:47.786326 24948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:34:47.786374 24948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:34:47.786433 24948 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:34:47.786471 24948 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:34:47.792140 24948 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:35209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:33519
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:33519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:34:47.793165 24948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:34:47.794163 24948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:34:47.802706 24956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:34:47.802743 24953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:34:47.802824 24954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:34:47.803005 24948 server_base.cc:1034] running on GCE node
I20260926 16:34:47.803512 24948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:34:47.804544 24948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:34:47.805739 24948 hybrid_clock.cc:648] HybridClock initialized: now 1790440487805728 us; error 25 us; skew 500 ppm
Sep 26 16:34:47 dist-test-slave-2zvq krb5kdc[24935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440487, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:34:47.808416 24948 init.cc:377] Logged in from keytab as kudu/127.11.8.254@KRBTEST.COM (short username kudu)
I20260926 16:34:47.809253 24948 webserver.cc:460] Webserver started at http://127.11.8.254:36157/ using document root <none> and password file <none>
I20260926 16:34:47.809794 24948 fs_manager.cc:359] Metadata directory not provided
I20260926 16:34:47.809885 24948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:34:47.810109 24948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:34:47.812132 24948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "83abcbf830ef4c77b0bb5cfe4ac24a2c"
format_stamp: "Formatted at 2026-09-26 16:34:47 on dist-test-slave-2zvq"
server_key: "481019d0a0969a0187c3e636f14b4065"
server_key_iv: "9d559ae691a8a1b44a3ea5edbe6926cc"
server_key_version: "encryptionkey@0"
I20260926 16:34:47.812614 24948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "83abcbf830ef4c77b0bb5cfe4ac24a2c"
format_stamp: "Formatted at 2026-09-26 16:34:47 on dist-test-slave-2zvq"
server_key: "481019d0a0969a0187c3e636f14b4065"
server_key_iv: "9d559ae691a8a1b44a3ea5edbe6926cc"
server_key_version: "encryptionkey@0"
I20260926 16:34:47.816246 24948 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 16:34:47.818560 24963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:47.819783 24948 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260926 16:34:47.819926 24948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "83abcbf830ef4c77b0bb5cfe4ac24a2c"
format_stamp: "Formatted at 2026-09-26 16:34:47 on dist-test-slave-2zvq"
server_key: "481019d0a0969a0187c3e636f14b4065"
server_key_iv: "9d559ae691a8a1b44a3ea5edbe6926cc"
server_key_version: "encryptionkey@0"
I20260926 16:34:47.820036 24948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:34:47.847142 24948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:34:47.850144 24948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:34:47.850404 24948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:34:47.857383 24948 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:33519
I20260926 16:34:47.857410 25015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:33519 every 8 connection(s)
I20260926 16:34:47.858361 24948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:34:47.858996 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 24948
I20260926 16:34:47.859148 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:34:47.859463 11299 external_mini_cluster.cc:1428] Setting key 623a33fa8abcb02bade9cc1cdb616a4f
I20260926 16:34:47.861616 25016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:34:47.866938 25016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83abcbf830ef4c77b0bb5cfe4ac24a2c: Bootstrap starting.
Sep 26 16:34:47 dist-test-slave-2zvq krb5kdc[24935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440487, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:34:47.869501 25016 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83abcbf830ef4c77b0bb5cfe4ac24a2c: Neither blocks nor log segments found. Creating new log.
I20260926 16:34:47.870215 25016 log.cc:824] T 00000000000000000000000000000000 P 83abcbf830ef4c77b0bb5cfe4ac24a2c: Log is configured to *not* fsync() on all Append() calls
I20260926 16:34:47.872025 25016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83abcbf830ef4c77b0bb5cfe4ac24a2c: No bootstrap required, opened a new log
I20260926 16:34:47.873042 25019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:47.860792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57432 (local address 127.11.8.254:33519)
0926 16:34:47.861248 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:47.861262 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:47.861298 (+ 36us) server_negotiation.cc:407] Connection header received
0926 16:34:47.861998 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:47.862016 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:47.862407 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:47.862734 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:34:47.863513 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:47.864486 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:47.865314 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:47.865772 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:47.865939 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:47.868544 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:34:47.868549 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:34:47.868561 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:34:47.868586 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:34:47.870307 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:47.870749 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:47.870762 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:47.870767 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:47.870869 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:47.871146 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:47.871149 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:47.871151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:47.871514 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:34:47.871718 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:47.871991 (+ 273us) server_negotiation.cc:299] Negotiation successful
0926 16:34:47.872378 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":145,"threads_started":1}
I20260926 16:34:47.875607 25016 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83abcbf830ef4c77b0bb5cfe4ac24a2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83abcbf830ef4c77b0bb5cfe4ac24a2c" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 33519 } }
I20260926 16:34:47.875818 11299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:34:47.875900 25016 raft_consensus.cc:374] T 00000000000000000000000000000000 P 83abcbf830ef4c77b0bb5cfe4ac24a2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:34:47.875983 25016 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83abcbf830ef4c77b0bb5cfe4ac24a2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83abcbf830ef4c77b0bb5cfe4ac24a2c, State: Initialized, Role: FOLLOWER
I20260926 16:34:47.876398 25016 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83abcbf830ef4c77b0bb5cfe4ac24a2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83abcbf830ef4c77b0bb5cfe4ac24a2c" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 33519 } }
I20260926 16:34:47.876552 25016 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83abcbf830ef4c77b0bb5cfe4ac24a2c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:34:47.876639 25016 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83abcbf830ef4c77b0bb5cfe4ac24a2c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:34:47.876745 25016 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83abcbf830ef4c77b0bb5cfe4ac24a2c [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:34:47.877750 25016 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83abcbf830ef4c77b0bb5cfe4ac24a2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83abcbf830ef4c77b0bb5cfe4ac24a2c" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 33519 } }
I20260926 16:34:47.878041 25016 leader_election.cc:302] T 00000000000000000000000000000000 P 83abcbf830ef4c77b0bb5cfe4ac24a2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83abcbf830ef4c77b0bb5cfe4ac24a2c; no voters:
I20260926 16:34:47.878329 25016 leader_election.cc:288] T 00000000000000000000000000000000 P 83abcbf830ef4c77b0bb5cfe4ac24a2c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:34:47.878445 25021 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83abcbf830ef4c77b0bb5cfe4ac24a2c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:34:47.878708 25021 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83abcbf830ef4c77b0bb5cfe4ac24a2c [term 1 LEADER]: Becoming Leader. State: Replica: 83abcbf830ef4c77b0bb5cfe4ac24a2c, State: Running, Role: LEADER
I20260926 16:34:47.879024 25021 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83abcbf830ef4c77b0bb5cfe4ac24a2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83abcbf830ef4c77b0bb5cfe4ac24a2c" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 33519 } }
I20260926 16:34:47.879550 25016 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83abcbf830ef4c77b0bb5cfe4ac24a2c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:34:47.880616 25022 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83abcbf830ef4c77b0bb5cfe4ac24a2c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "83abcbf830ef4c77b0bb5cfe4ac24a2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83abcbf830ef4c77b0bb5cfe4ac24a2c" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 33519 } } }
I20260926 16:34:47.880741 25022 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83abcbf830ef4c77b0bb5cfe4ac24a2c [sys.catalog]: This master's current role is: LEADER
I20260926 16:34:47.881112 25030 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:34:47.881072 25023 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83abcbf830ef4c77b0bb5cfe4ac24a2c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83abcbf830ef4c77b0bb5cfe4ac24a2c. Latest consensus state: current_term: 1 leader_uuid: "83abcbf830ef4c77b0bb5cfe4ac24a2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83abcbf830ef4c77b0bb5cfe4ac24a2c" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 33519 } } }
I20260926 16:34:47.881269 25023 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83abcbf830ef4c77b0bb5cfe4ac24a2c [sys.catalog]: This master's current role is: LEADER
I20260926 16:34:47.884231 25030 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:34:47.889125 25030 catalog_manager.cc:1347] Generated new cluster ID: 04b469b2e4f94d359cfcde77d44dd6bd
I20260926 16:34:47.889186 25030 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:34:47.909152 25030 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:34:47.910084 25030 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:34:47.915484 25030 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83abcbf830ef4c77b0bb5cfe4ac24a2c: Generated new TSK 0
I20260926 16:34:47.916160 25030 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:34:47.931704 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 24948
2026-09-26T16:34:47Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1420 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:34:47 dist-test-slave-2zvq krb5kdc[25043](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:34:47 dist-test-slave-2zvq krb5kdc[25043](info): set up 2 sockets
Sep 26 16:34:47 dist-test-slave-2zvq krb5kdc[25043](info): commencing operation
krb5kdc: starting...
W20260926 16:34:48.976650 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:34:49 dist-test-slave-2zvq krb5kdc[25043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:34:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:34:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
I20260926 16:34:49.098186 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:46389
--webserver_interface=127.11.8.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:39665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:46389
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:34:49.207628 25056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:34:49.207880 25056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:34:49.207919 25056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:34:49.211817 25056 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:34:49.211875 25056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:34:49.211894 25056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:34:49.211910 25056 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:34:49.211925 25056 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:34:49.217190 25056 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:39665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:46389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:46389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:34:49.218180 25056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:34:49.219244 25056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:34:49.227909 25062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:34:49.227859 25061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:34:49.228142 25056 server_base.cc:1034] running on GCE node
W20260926 16:34:49.228008 25064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:34:49.228825 25056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:34:49.229893 25056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:34:49.231077 25056 hybrid_clock.cc:648] HybridClock initialized: now 1790440489231049 us; error 42 us; skew 500 ppm
Sep 26 16:34:49 dist-test-slave-2zvq krb5kdc[25043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440489, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:34:49.241364 25056 init.cc:377] Logged in from keytab as kudu/127.11.8.254@KRBTEST.COM (short username kudu)
I20260926 16:34:49.242390 25056 webserver.cc:460] Webserver started at http://127.11.8.254:34519/ using document root <none> and password file <none>
I20260926 16:34:49.242954 25056 fs_manager.cc:359] Metadata directory not provided
I20260926 16:34:49.243043 25056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:34:49.243278 25056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:34:49.245306 25056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "620a6df881a040c7b5d5266f54fb2158"
format_stamp: "Formatted at 2026-09-26 16:34:49 on dist-test-slave-2zvq"
server_key: "7d6281c966055fc1ecb90d7a8cb0a8b1"
server_key_iv: "d8240f03fa1092e08882b6407ccad068"
server_key_version: "encryptionkey@0"
I20260926 16:34:49.245850 25056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "620a6df881a040c7b5d5266f54fb2158"
format_stamp: "Formatted at 2026-09-26 16:34:49 on dist-test-slave-2zvq"
server_key: "7d6281c966055fc1ecb90d7a8cb0a8b1"
server_key_iv: "d8240f03fa1092e08882b6407ccad068"
server_key_version: "encryptionkey@0"
I20260926 16:34:49.249487 25056 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260926 16:34:49.251895 25071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:49.253069 25056 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 16:34:49.253230 25056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "620a6df881a040c7b5d5266f54fb2158"
format_stamp: "Formatted at 2026-09-26 16:34:49 on dist-test-slave-2zvq"
server_key: "7d6281c966055fc1ecb90d7a8cb0a8b1"
server_key_iv: "d8240f03fa1092e08882b6407ccad068"
server_key_version: "encryptionkey@0"
I20260926 16:34:49.253340 25056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:34:49.266080 25056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:34:49.269304 25056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:34:49.269503 25056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:34:49.276479 25056 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:46389
I20260926 16:34:49.276743 25123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:46389 every 8 connection(s)
I20260926 16:34:49.277446 25056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:34:49.280107 25124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:34:49.284248 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 25056
I20260926 16:34:49.284397 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:34:49.284657 11299 external_mini_cluster.cc:1428] Setting key 5748abe34c2f75ebc6932750a69a829b
I20260926 16:34:49.285647 25124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158: Bootstrap starting.
I20260926 16:34:49.288213 25124 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158: Neither blocks nor log segments found. Creating new log.
I20260926 16:34:49.289033 25124 log.cc:824] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158: Log is configured to *not* fsync() on all Append() calls
I20260926 16:34:49.291013 25124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158: No bootstrap required, opened a new log
Sep 26 16:34:49 dist-test-slave-2zvq krb5kdc[25043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:34:49.295006 25124 raft_consensus.cc:348] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "620a6df881a040c7b5d5266f54fb2158" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 46389 } }
I20260926 16:34:49.295292 25124 raft_consensus.cc:374] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:34:49.295358 25124 raft_consensus.cc:729] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 620a6df881a040c7b5d5266f54fb2158, State: Initialized, Role: FOLLOWER
I20260926 16:34:49.295853 25124 consensus_queue.cc:260] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "620a6df881a040c7b5d5266f54fb2158" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 46389 } }
I20260926 16:34:49.296010 25124 raft_consensus.cc:388] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:34:49.296074 25124 raft_consensus.cc:482] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:34:49.296167 25124 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:34:49.297262 25124 raft_consensus.cc:504] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "620a6df881a040c7b5d5266f54fb2158" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 46389 } }
I20260926 16:34:49.297607 25124 leader_election.cc:302] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 620a6df881a040c7b5d5266f54fb2158; no voters:
I20260926 16:34:49.297926 25124 leader_election.cc:288] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:34:49.298033 25129 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:34:49.298259 25129 raft_consensus.cc:686] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [term 1 LEADER]: Becoming Leader. State: Replica: 620a6df881a040c7b5d5266f54fb2158, State: Running, Role: LEADER
I20260926 16:34:49.298592 25129 consensus_queue.cc:237] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "620a6df881a040c7b5d5266f54fb2158" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 46389 } }
I20260926 16:34:49.299292 25124 sys_catalog.cc:564] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:34:49.300005 25127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:49.285987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47836 (local address 127.11.8.254:46389)
0926 16:34:49.286396 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:49.286414 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:49.286453 (+ 39us) server_negotiation.cc:407] Connection header received
0926 16:34:49.287286 (+ 833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:49.287309 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:49.287692 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:49.288039 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:34:49.288778 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:49.290126 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:49.291012 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:49.291677 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:49.291852 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:49.294496 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:34:49.294502 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:34:49.294517 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:34:49.294547 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:34:49.296507 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:49.296898 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:49.296906 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:49.296912 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:49.297050 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:49.297436 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:49.297440 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:49.297442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:49.297956 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:34:49.298202 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:49.298696 (+ 494us) server_negotiation.cc:299] Negotiation successful
0926 16:34:49.299018 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":102,"threads_started":1}
I20260926 16:34:49.300570 25131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 620a6df881a040c7b5d5266f54fb2158. Latest consensus state: current_term: 1 leader_uuid: "620a6df881a040c7b5d5266f54fb2158" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "620a6df881a040c7b5d5266f54fb2158" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 46389 } } }
I20260926 16:34:49.300619 25130 sys_catalog.cc:455] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "620a6df881a040c7b5d5266f54fb2158" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "620a6df881a040c7b5d5266f54fb2158" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 46389 } } }
I20260926 16:34:49.300693 25131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [sys.catalog]: This master's current role is: LEADER
I20260926 16:34:49.300720 25130 sys_catalog.cc:458] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [sys.catalog]: This master's current role is: LEADER
I20260926 16:34:49.301081 25139 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:34:49.304114 25139 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:34:49.309168 25139 catalog_manager.cc:1347] Generated new cluster ID: bb4e410281e241999082312ec85add52
I20260926 16:34:49.309233 25139 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:34:49.329408 25139 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:34:49.330361 25139 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:34:49.338025 25139 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158: Generated new TSK 0
I20260926 16:34:49.338810 25139 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.8.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab.
Entry for principal kudu/127.11.8.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab.
I20260926 16:34:49.365736 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.193:0
--local_ip_for_outbound_sockets=127.11.8.193
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:46389
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:39665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:34:49.492034 25150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:34:49.492316 25150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:34:49.492393 25150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:34:49.496500 25150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:34:49.496578 25150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:34:49.496668 25150 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.193
I20260926 16:34:49.502247 25150 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:39665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:46389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25150
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.193
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:34:49.503281 25150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:34:49.504262 25150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:34:49.513381 25156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:34:49.513445 25158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:34:49.513454 25155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:34:49.513631 25150 server_base.cc:1034] running on GCE node
I20260926 16:34:49.514015 25150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:34:49.514663 25150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:34:49.515836 25150 hybrid_clock.cc:648] HybridClock initialized: now 1790440489515803 us; error 41 us; skew 500 ppm
Sep 26 16:34:49 dist-test-slave-2zvq krb5kdc[25043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440489, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:34:49.518460 25150 init.cc:377] Logged in from keytab as kudu/127.11.8.193@KRBTEST.COM (short username kudu)
I20260926 16:34:49.519196 25150 webserver.cc:460] Webserver started at http://127.11.8.193:46449/ using document root <none> and password file <none>
I20260926 16:34:49.519675 25150 fs_manager.cc:359] Metadata directory not provided
I20260926 16:34:49.519731 25150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:34:49.519891 25150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:34:49.522033 25150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/data/instance:
uuid: "7a6306c756d14d899623fc8b45e187ce"
format_stamp: "Formatted at 2026-09-26 16:34:49 on dist-test-slave-2zvq"
server_key: "0c4b5b14b95a43edce8654daee6cf0fa"
server_key_iv: "e8d2f45270d6fa44001e5e8f6d12009a"
server_key_version: "encryptionkey@0"
I20260926 16:34:49.522578 25150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance:
uuid: "7a6306c756d14d899623fc8b45e187ce"
format_stamp: "Formatted at 2026-09-26 16:34:49 on dist-test-slave-2zvq"
server_key: "0c4b5b14b95a43edce8654daee6cf0fa"
server_key_iv: "e8d2f45270d6fa44001e5e8f6d12009a"
server_key_version: "encryptionkey@0"
I20260926 16:34:49.525928 25150 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:34:49.528230 25165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:49.529393 25150 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:34:49.529614 25150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/wal
uuid: "7a6306c756d14d899623fc8b45e187ce"
format_stamp: "Formatted at 2026-09-26 16:34:49 on dist-test-slave-2zvq"
server_key: "0c4b5b14b95a43edce8654daee6cf0fa"
server_key_iv: "e8d2f45270d6fa44001e5e8f6d12009a"
server_key_version: "encryptionkey@0"
I20260926 16:34:49.529712 25150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:34:49.557650 25150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:34:49.561007 25150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:34:49.561201 25150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:34:49.561892 25150 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:34:49.562981 25150 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:34:49.563040 25150 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:49.563092 25150 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:34:49.563124 25150 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:49.572142 25150 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.193:46507
I20260926 16:34:49.572170 25278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.193:46507 every 8 connection(s)
I20260926 16:34:49.573005 25150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
Sep 26 16:34:49 dist-test-slave-2zvq krb5kdc[25043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440489, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:34:49.582603 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 25150
I20260926 16:34:49.582772 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance
I20260926 16:34:49.583073 11299 external_mini_cluster.cc:1428] Setting key 2661713e937069c7e4ac7ef0c446dad0
I20260926 16:34:49.586520 25127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:49.574751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:38391 (local address 127.11.8.254:46389)
0926 16:34:49.574877 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:49.574880 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:49.575666 (+ 786us) server_negotiation.cc:407] Connection header received
0926 16:34:49.577002 (+ 1336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:49.577005 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:49.577058 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:49.577139 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:34:49.578254 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:49.578813 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:49.579949 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:49.580226 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:49.580339 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:49.582777 (+ 2438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:34:49.582779 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:34:49.582782 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:34:49.582803 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:34:49.584477 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:49.584961 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:49.584966 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:49.584968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:49.585054 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:49.585396 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:49.585401 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:49.585403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:49.585581 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:34:49.585690 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:49.586186 (+ 496us) server_negotiation.cc:299] Negotiation successful
0926 16:34:49.586348 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 16:34:49.587288 25281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:49.575124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.193:38391)
0926 16:34:49.575531 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:49.575573 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 16:34:49.576798 (+ 1225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:34:49.577309 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:34:49.577316 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:34:49.577661 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:34:49.578112 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:49.578124 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:49.578939 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:49.578942 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:49.579825 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:49.579834 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:49.580387 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:49.580391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:49.580550 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:49.581083 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:34:49.581099 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:34:49.582566 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:34:49.584577 (+ 2011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:49.584582 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:49.584598 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:49.584859 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:49.585156 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:49.585159 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:49.585161 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:49.585304 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:49.585697 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:34:49.585705 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:34:49.585953 (+ 248us) client_negotiation.cc:769] Sending connection context
0926 16:34:49.586137 (+ 184us) client_negotiation.cc:240] Negotiation successful
0926 16:34:49.586475 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":96,"threads_started":1}
I20260926 16:34:49.588608 25279 heartbeater.cc:347] Connected to a master server at 127.11.8.254:46389
I20260926 16:34:49.588855 25279 heartbeater.cc:464] Registering TS with master...
I20260926 16:34:49.589318 25279 heartbeater.cc:511] Master 127.11.8.254:46389 requested a full tablet report, sending...
I20260926 16:34:49.590843 25088 ts_manager.cc:194] Registered new tserver with Master: 7a6306c756d14d899623fc8b45e187ce (127.11.8.193:46507)
I20260926 16:34:49.591948 25088 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.193@KRBTEST.COM'} at 127.11.8.193:38391
WARNING: no policy specified for kudu/127.11.8.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab.
Entry for principal kudu/127.11.8.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab.
I20260926 16:34:49.612900 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.194:0
--local_ip_for_outbound_sockets=127.11.8.194
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:46389
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:39665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:34:49.722962 25284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:34:49.723194 25284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:34:49.723268 25284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:34:49.727264 25284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:34:49.727313 25284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:34:49.727406 25284 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.194
I20260926 16:34:49.732636 25284 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:39665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:46389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.194
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:34:49.733670 25284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:34:49.734758 25284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:34:49.744316 25290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:34:49.744320 25289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:34:49.744333 25292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:34:49.744534 25284 server_base.cc:1034] running on GCE node
I20260926 16:34:49.744948 25284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:34:49.745499 25284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:34:49.746654 25284 hybrid_clock.cc:648] HybridClock initialized: now 1790440489746638 us; error 28 us; skew 500 ppm
Sep 26 16:34:49 dist-test-slave-2zvq krb5kdc[25043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440489, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:34:49.749261 25284 init.cc:377] Logged in from keytab as kudu/127.11.8.194@KRBTEST.COM (short username kudu)
I20260926 16:34:49.750106 25284 webserver.cc:460] Webserver started at http://127.11.8.194:45385/ using document root <none> and password file <none>
I20260926 16:34:49.750679 25284 fs_manager.cc:359] Metadata directory not provided
I20260926 16:34:49.750746 25284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:34:49.750948 25284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:34:49.752972 25284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/data/instance:
uuid: "9f3248e0f8d444edb82c65ff34e884a9"
format_stamp: "Formatted at 2026-09-26 16:34:49 on dist-test-slave-2zvq"
server_key: "cc4d9911c1a7a00ea1f1d467d1aa0c56"
server_key_iv: "ec7e67e6371902dcb3a82eeb64c2d647"
server_key_version: "encryptionkey@0"
I20260926 16:34:49.753561 25284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance:
uuid: "9f3248e0f8d444edb82c65ff34e884a9"
format_stamp: "Formatted at 2026-09-26 16:34:49 on dist-test-slave-2zvq"
server_key: "cc4d9911c1a7a00ea1f1d467d1aa0c56"
server_key_iv: "ec7e67e6371902dcb3a82eeb64c2d647"
server_key_version: "encryptionkey@0"
I20260926 16:34:49.757213 25284 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260926 16:34:49.759718 25299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:49.760905 25284 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 16:34:49.761035 25284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/wal
uuid: "9f3248e0f8d444edb82c65ff34e884a9"
format_stamp: "Formatted at 2026-09-26 16:34:49 on dist-test-slave-2zvq"
server_key: "cc4d9911c1a7a00ea1f1d467d1aa0c56"
server_key_iv: "ec7e67e6371902dcb3a82eeb64c2d647"
server_key_version: "encryptionkey@0"
I20260926 16:34:49.761147 25284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:34:49.776495 25284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:34:49.779426 25284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:34:49.779627 25284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:34:49.780200 25284 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:34:49.780999 25284 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:34:49.781069 25284 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:49.781141 25284 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:34:49.781185 25284 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:49.790320 25284 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.194:34133
I20260926 16:34:49.790346 25412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.194:34133 every 8 connection(s)
I20260926 16:34:49.791162 25284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
I20260926 16:34:49.799360 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 25284
I20260926 16:34:49.799482 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance
I20260926 16:34:49.799786 11299 external_mini_cluster.cc:1428] Setting key e667b33beb8d8a248bdbfe4dfb80267c
Sep 26 16:34:49 dist-test-slave-2zvq krb5kdc[25043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440489, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:34:49.805238 25127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:49.792822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:46871 (local address 127.11.8.254:46389)
0926 16:34:49.792948 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:49.792952 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:49.793769 (+ 817us) server_negotiation.cc:407] Connection header received
0926 16:34:49.795148 (+ 1379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:49.795151 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:49.795196 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:49.795263 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:34:49.796412 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:49.796962 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:49.798110 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:49.798388 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:49.798465 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:49.801682 (+ 3217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:34:49.801686 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:34:49.801689 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:34:49.801716 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:34:49.803165 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:49.803689 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:49.803693 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:49.803695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:49.803777 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:49.804152 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:49.804160 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:49.804161 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:49.804325 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:34:49.804445 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:49.804941 (+ 496us) server_negotiation.cc:299] Negotiation successful
0926 16:34:49.805057 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 16:34:49.806048 25415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:49.793172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.194:46871)
0926 16:34:49.793624 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:49.793663 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 16:34:49.794947 (+ 1284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:34:49.795455 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:34:49.795461 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:34:49.795821 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:34:49.796279 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:49.796292 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:49.797103 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:49.797106 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:49.797997 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:49.798007 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:49.798513 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:49.798516 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:49.798672 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:49.799629 (+ 957us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:34:49.799651 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:34:49.801500 (+ 1849us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:34:49.803284 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:49.803289 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:49.803303 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:49.803574 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:49.803879 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:49.803882 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:49.803884 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:49.804023 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:49.804436 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:34:49.804442 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:34:49.804701 (+ 259us) client_negotiation.cc:769] Sending connection context
0926 16:34:49.804894 (+ 193us) client_negotiation.cc:240] Negotiation successful
0926 16:34:49.805201 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":95,"threads_started":1}
I20260926 16:34:49.807323 25413 heartbeater.cc:347] Connected to a master server at 127.11.8.254:46389
I20260926 16:34:49.807571 25413 heartbeater.cc:464] Registering TS with master...
I20260926 16:34:49.808034 25413 heartbeater.cc:511] Master 127.11.8.254:46389 requested a full tablet report, sending...
I20260926 16:34:49.809098 25088 ts_manager.cc:194] Registered new tserver with Master: 9f3248e0f8d444edb82c65ff34e884a9 (127.11.8.194:34133)
WARNING: no policy specified for kudu/127.11.8.195@KRBTEST.COM; defaulting to no policy
I20260926 16:34:49.809643 25088 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.194@KRBTEST.COM'} at 127.11.8.194:46871
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab.
Entry for principal kudu/127.11.8.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab.
I20260926 16:34:49.829046 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.195:0
--local_ip_for_outbound_sockets=127.11.8.195
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:46389
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:39665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:34:49.937842 25418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:34:49.938081 25418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:34:49.938122 25418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:34:49.942041 25418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:34:49.942092 25418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:34:49.942148 25418 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.195
I20260926 16:34:49.947242 25418 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:39665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:46389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.195
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:34:49.948151 25418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:34:49.949064 25418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:34:49.958380 25423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:34:49.958391 25424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:34:49.958530 25418 server_base.cc:1034] running on GCE node
W20260926 16:34:49.958391 25426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:34:49.959069 25418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:34:49.959613 25418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:34:49.960803 25418 hybrid_clock.cc:648] HybridClock initialized: now 1790440489960770 us; error 47 us; skew 500 ppm
Sep 26 16:34:49 dist-test-slave-2zvq krb5kdc[25043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440489, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:34:49.963546 25418 init.cc:377] Logged in from keytab as kudu/127.11.8.195@KRBTEST.COM (short username kudu)
I20260926 16:34:49.964507 25418 webserver.cc:460] Webserver started at http://127.11.8.195:37475/ using document root <none> and password file <none>
I20260926 16:34:49.965067 25418 fs_manager.cc:359] Metadata directory not provided
I20260926 16:34:49.965155 25418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:34:49.965322 25418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:34:49.967265 25418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/data/instance:
uuid: "b3760e55349542ddaf855c0fe60d782c"
format_stamp: "Formatted at 2026-09-26 16:34:49 on dist-test-slave-2zvq"
server_key: "9c82336090ac85ab12e89d56946fbad8"
server_key_iv: "2b166f5ad0f0fc921589dbc5bb521f90"
server_key_version: "encryptionkey@0"
I20260926 16:34:49.967747 25418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance:
uuid: "b3760e55349542ddaf855c0fe60d782c"
format_stamp: "Formatted at 2026-09-26 16:34:49 on dist-test-slave-2zvq"
server_key: "9c82336090ac85ab12e89d56946fbad8"
server_key_iv: "2b166f5ad0f0fc921589dbc5bb521f90"
server_key_version: "encryptionkey@0"
I20260926 16:34:49.971104 25418 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 16:34:49.973277 25433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:49.974565 25418 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:34:49.974659 25418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/wal
uuid: "b3760e55349542ddaf855c0fe60d782c"
format_stamp: "Formatted at 2026-09-26 16:34:49 on dist-test-slave-2zvq"
server_key: "9c82336090ac85ab12e89d56946fbad8"
server_key_iv: "2b166f5ad0f0fc921589dbc5bb521f90"
server_key_version: "encryptionkey@0"
I20260926 16:34:49.974738 25418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:34:50.005949 25418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:34:50.008975 25418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:34:50.009137 25418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:34:50.009662 25418 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:34:50.010485 25418 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:34:50.010535 25418 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:50.010574 25418 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:34:50.010589 25418 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:50.019687 25418 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.195:33957
I20260926 16:34:50.019716 25546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.195:33957 every 8 connection(s)
I20260926 16:34:50.020540 25418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
I20260926 16:34:50.025404 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 25418
I20260926 16:34:50.025542 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance
I20260926 16:34:50.025820 11299 external_mini_cluster.cc:1428] Setting key b6a8194aba86af8138c2b77cbe4590f2
Sep 26 16:34:50 dist-test-slave-2zvq krb5kdc[25043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440489, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:34:50.035058 25127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:50.022475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:45019 (local address 127.11.8.254:46389)
0926 16:34:50.022627 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:50.022630 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:50.023481 (+ 851us) server_negotiation.cc:407] Connection header received
0926 16:34:50.024873 (+ 1392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:50.024877 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:50.024934 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:50.025043 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:34:50.026416 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.027074 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.028587 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.028813 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.029008 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:50.031547 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:34:50.031550 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:34:50.031552 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:34:50.031576 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:34:50.032993 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:50.033593 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:50.033596 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:50.033597 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:50.033677 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:50.033995 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:50.033997 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:50.033998 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:50.034198 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:34:50.034344 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:50.034830 (+ 486us) server_negotiation.cc:299] Negotiation successful
0926 16:34:50.034928 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 16:34:50.035861 25549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:50.022873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.195:45019)
0926 16:34:50.023330 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:50.023370 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 16:34:50.024653 (+ 1283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:34:50.025226 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:34:50.025233 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:34:50.025665 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:34:50.026244 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:50.026263 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.027229 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.027233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:50.028448 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:50.028458 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.029038 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.029041 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:50.029208 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:50.029839 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:34:50.029855 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:34:50.031402 (+ 1547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:34:50.033111 (+ 1709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:50.033116 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:50.033126 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:50.033460 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:50.033769 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:50.033772 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:50.033773 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:50.033903 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:50.034352 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:34:50.034358 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:34:50.034580 (+ 222us) client_negotiation.cc:769] Sending connection context
0926 16:34:50.034734 (+ 154us) client_negotiation.cc:240] Negotiation successful
0926 16:34:50.034995 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":101,"threads_started":1}
I20260926 16:34:50.037011 25547 heartbeater.cc:347] Connected to a master server at 127.11.8.254:46389
I20260926 16:34:50.037263 25547 heartbeater.cc:464] Registering TS with master...
I20260926 16:34:50.037729 25547 heartbeater.cc:511] Master 127.11.8.254:46389 requested a full tablet report, sending...
I20260926 16:34:50.039022 25088 ts_manager.cc:194] Registered new tserver with Master: b3760e55349542ddaf855c0fe60d782c (127.11.8.195:33957)
I20260926 16:34:50.039762 25088 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.195@KRBTEST.COM'} at 127.11.8.195:45019
I20260926 16:34:50.039866 11299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 16:34:50.049248 25127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:50.042009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47840 (local address 127.11.8.254:46389)
0926 16:34:50.042147 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:50.042152 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:50.042380 (+ 228us) server_negotiation.cc:407] Connection header received
0926 16:34:50.042466 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:50.042468 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:50.042504 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:50.042572 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:34:50.043219 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.043731 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.044531 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.044749 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.044825 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:50.046088 (+ 1263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:34:50.046091 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:34:50.046093 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:34:50.046147 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:34:50.047659 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:50.048048 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:50.048051 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:50.048052 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:50.048154 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:50.048438 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:50.048441 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:50.048442 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:50.048648 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:34:50.048760 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:50.048939 (+ 179us) server_negotiation.cc:299] Negotiation successful
0926 16:34:50.049026 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 16:34:50.052081 25088 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47840:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:34:50.055028 25088 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:34:50.069423 25562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:50.063986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34368 (local address 127.11.8.195:33957)
0926 16:34:50.064490 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:50.064503 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:50.064834 (+ 331us) server_negotiation.cc:407] Connection header received
0926 16:34:50.064952 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:50.064958 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:50.065132 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:50.065292 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:34:50.065554 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.066624 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.068209 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.068781 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.068902 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:50.069042 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:50.069196 (+ 154us) server_negotiation.cc:299] Negotiation successful
0926 16:34:50.069280 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":173,"threads_started":1}
I20260926 16:34:50.069413 25561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:50.063357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:33957 (local address 127.0.0.1:34368)
0926 16:34:50.064724 (+ 1367us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:50.064737 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:34:50.064838 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:34:50.065285 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:34:50.065288 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:34:50.065306 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:34:50.065440 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:50.065447 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.066758 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.066761 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:50.068083 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:50.068095 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.068902 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.068905 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:50.069007 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:50.069040 (+ 33us) client_negotiation.cc:769] Sending connection context
0926 16:34:50.069151 (+ 111us) client_negotiation.cc:240] Negotiation successful
0926 16:34:50.069270 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1304,"mutex_wait_us":26,"thread_start_us":107,"threads_started":1}
I20260926 16:34:50.069846 25558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:50.063922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:34133 (local address 127.0.0.1:32850)
0926 16:34:50.065007 (+ 1085us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:50.065021 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:34:50.065094 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:34:50.065755 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:34:50.065758 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:34:50.065774 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:34:50.065874 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:50.065879 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.067086 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.067089 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:50.067914 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:50.067924 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.069121 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.069125 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:50.069240 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:50.069263 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 16:34:50.069339 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 16:34:50.069418 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1029,"mutex_wait_us":2,"thread_start_us":101,"threads_started":1}
I20260926 16:34:50.070101 25560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:50.063270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:46507 (local address 127.0.0.1:44802)
0926 16:34:50.063847 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:50.063897 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 16:34:50.064089 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:34:50.064914 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:34:50.064921 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:34:50.064946 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:34:50.065069 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:50.065075 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.066602 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.066605 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:50.068235 (+ 1630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:50.068240 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.069405 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.069408 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:50.069627 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:50.069649 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 16:34:50.069730 (+ 81us) client_negotiation.cc:240] Negotiation successful
0926 16:34:50.069803 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"mutex_wait_us":84,"thread_start_us":57,"threads_started":1}
I20260926 16:34:50.070387 25559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:50.063384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44802 (local address 127.11.8.193:46507)
0926 16:34:50.064377 (+ 993us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:50.064394 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:50.064416 (+ 22us) server_negotiation.cc:407] Connection header received
0926 16:34:50.064499 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:50.064505 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:50.064727 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:50.064887 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:34:50.065212 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.066443 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.068345 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.069123 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.069223 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:50.069350 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:50.070025 (+ 675us) server_negotiation.cc:299] Negotiation successful
0926 16:34:50.070121 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":889,"thread_start_us":53,"threads_started":1}
I20260926 16:34:50.070456 25563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:50.064384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32850 (local address 127.11.8.194:34133)
0926 16:34:50.065253 (+ 869us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:50.065265 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:50.065285 (+ 20us) server_negotiation.cc:407] Connection header received
0926 16:34:50.065352 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:50.065356 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:50.065532 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:50.065701 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:34:50.065986 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.066943 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.068045 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.068807 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.068918 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:50.069054 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:50.069496 (+ 442us) server_negotiation.cc:299] Negotiation successful
0926 16:34:50.069614 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":745,"thread_start_us":92,"threads_started":1}
I20260926 16:34:50.072613 25347 tablet_service.cc:1467] Processing CreateTablet for tablet b899f5bd8cb444b6b077d07050a01ebf (DEFAULT_TABLE table=test-table [id=5c76558d1226418da10f82508e2b3370]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:34:50.072624 25481 tablet_service.cc:1467] Processing CreateTablet for tablet b899f5bd8cb444b6b077d07050a01ebf (DEFAULT_TABLE table=test-table [id=5c76558d1226418da10f82508e2b3370]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:34:50.072965 25213 tablet_service.cc:1467] Processing CreateTablet for tablet b899f5bd8cb444b6b077d07050a01ebf (DEFAULT_TABLE table=test-table [id=5c76558d1226418da10f82508e2b3370]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:34:50.073621 25347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b899f5bd8cb444b6b077d07050a01ebf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:34:50.073623 25481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b899f5bd8cb444b6b077d07050a01ebf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:34:50.073796 25213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b899f5bd8cb444b6b077d07050a01ebf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:34:50.079656 25564 tablet_bootstrap.cc:492] T b899f5bd8cb444b6b077d07050a01ebf P 9f3248e0f8d444edb82c65ff34e884a9: Bootstrap starting.
I20260926 16:34:50.081192 25566 tablet_bootstrap.cc:492] T b899f5bd8cb444b6b077d07050a01ebf P b3760e55349542ddaf855c0fe60d782c: Bootstrap starting.
I20260926 16:34:50.082247 25564 tablet_bootstrap.cc:654] T b899f5bd8cb444b6b077d07050a01ebf P 9f3248e0f8d444edb82c65ff34e884a9: Neither blocks nor log segments found. Creating new log.
I20260926 16:34:50.082592 25565 tablet_bootstrap.cc:492] T b899f5bd8cb444b6b077d07050a01ebf P 7a6306c756d14d899623fc8b45e187ce: Bootstrap starting.
I20260926 16:34:50.083073 25564 log.cc:824] T b899f5bd8cb444b6b077d07050a01ebf P 9f3248e0f8d444edb82c65ff34e884a9: Log is configured to *not* fsync() on all Append() calls
I20260926 16:34:50.083513 25566 tablet_bootstrap.cc:654] T b899f5bd8cb444b6b077d07050a01ebf P b3760e55349542ddaf855c0fe60d782c: Neither blocks nor log segments found. Creating new log.
I20260926 16:34:50.084296 25566 log.cc:824] T b899f5bd8cb444b6b077d07050a01ebf P b3760e55349542ddaf855c0fe60d782c: Log is configured to *not* fsync() on all Append() calls
I20260926 16:34:50.084774 25565 tablet_bootstrap.cc:654] T b899f5bd8cb444b6b077d07050a01ebf P 7a6306c756d14d899623fc8b45e187ce: Neither blocks nor log segments found. Creating new log.
I20260926 16:34:50.085413 25564 tablet_bootstrap.cc:492] T b899f5bd8cb444b6b077d07050a01ebf P 9f3248e0f8d444edb82c65ff34e884a9: No bootstrap required, opened a new log
I20260926 16:34:50.085611 25564 ts_tablet_manager.cc:1397] T b899f5bd8cb444b6b077d07050a01ebf P 9f3248e0f8d444edb82c65ff34e884a9: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260926 16:34:50.085829 25565 log.cc:824] T b899f5bd8cb444b6b077d07050a01ebf P 7a6306c756d14d899623fc8b45e187ce: Log is configured to *not* fsync() on all Append() calls
I20260926 16:34:50.087491 25565 tablet_bootstrap.cc:492] T b899f5bd8cb444b6b077d07050a01ebf P 7a6306c756d14d899623fc8b45e187ce: No bootstrap required, opened a new log
I20260926 16:34:50.087666 25565 ts_tablet_manager.cc:1397] T b899f5bd8cb444b6b077d07050a01ebf P 7a6306c756d14d899623fc8b45e187ce: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260926 16:34:50.088757 25564 raft_consensus.cc:348] T b899f5bd8cb444b6b077d07050a01ebf P 9f3248e0f8d444edb82c65ff34e884a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a6306c756d14d899623fc8b45e187ce" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46507 } } peers { permanent_uuid: "b3760e55349542ddaf855c0fe60d782c" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33957 } } peers { permanent_uuid: "9f3248e0f8d444edb82c65ff34e884a9" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34133 } }
I20260926 16:34:50.088959 25564 raft_consensus.cc:374] T b899f5bd8cb444b6b077d07050a01ebf P 9f3248e0f8d444edb82c65ff34e884a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:34:50.089016 25564 raft_consensus.cc:729] T b899f5bd8cb444b6b077d07050a01ebf P 9f3248e0f8d444edb82c65ff34e884a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f3248e0f8d444edb82c65ff34e884a9, State: Initialized, Role: FOLLOWER
I20260926 16:34:50.089453 25564 consensus_queue.cc:260] T b899f5bd8cb444b6b077d07050a01ebf P 9f3248e0f8d444edb82c65ff34e884a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a6306c756d14d899623fc8b45e187ce" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46507 } } peers { permanent_uuid: "b3760e55349542ddaf855c0fe60d782c" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33957 } } peers { permanent_uuid: "9f3248e0f8d444edb82c65ff34e884a9" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34133 } }
I20260926 16:34:50.090082 25413 heartbeater.cc:503] Master 127.11.8.254:46389 was elected leader, sending a full tablet report...
I20260926 16:34:50.090436 25564 ts_tablet_manager.cc:1428] T b899f5bd8cb444b6b077d07050a01ebf P 9f3248e0f8d444edb82c65ff34e884a9: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260926 16:34:50.090576 25565 raft_consensus.cc:348] T b899f5bd8cb444b6b077d07050a01ebf P 7a6306c756d14d899623fc8b45e187ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a6306c756d14d899623fc8b45e187ce" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46507 } } peers { permanent_uuid: "b3760e55349542ddaf855c0fe60d782c" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33957 } } peers { permanent_uuid: "9f3248e0f8d444edb82c65ff34e884a9" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34133 } }
I20260926 16:34:50.090821 25565 raft_consensus.cc:374] T b899f5bd8cb444b6b077d07050a01ebf P 7a6306c756d14d899623fc8b45e187ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:34:50.091048 25565 raft_consensus.cc:729] T b899f5bd8cb444b6b077d07050a01ebf P 7a6306c756d14d899623fc8b45e187ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a6306c756d14d899623fc8b45e187ce, State: Initialized, Role: FOLLOWER
I20260926 16:34:50.091456 25565 consensus_queue.cc:260] T b899f5bd8cb444b6b077d07050a01ebf P 7a6306c756d14d899623fc8b45e187ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a6306c756d14d899623fc8b45e187ce" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46507 } } peers { permanent_uuid: "b3760e55349542ddaf855c0fe60d782c" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33957 } } peers { permanent_uuid: "9f3248e0f8d444edb82c65ff34e884a9" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34133 } }
I20260926 16:34:50.092094 25279 heartbeater.cc:503] Master 127.11.8.254:46389 was elected leader, sending a full tablet report...
I20260926 16:34:50.092456 25565 ts_tablet_manager.cc:1428] T b899f5bd8cb444b6b077d07050a01ebf P 7a6306c756d14d899623fc8b45e187ce: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260926 16:34:50.094993 25566 tablet_bootstrap.cc:492] T b899f5bd8cb444b6b077d07050a01ebf P b3760e55349542ddaf855c0fe60d782c: No bootstrap required, opened a new log
I20260926 16:34:50.095220 25566 ts_tablet_manager.cc:1397] T b899f5bd8cb444b6b077d07050a01ebf P b3760e55349542ddaf855c0fe60d782c: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.000s
I20260926 16:34:50.099402 25566 raft_consensus.cc:348] T b899f5bd8cb444b6b077d07050a01ebf P b3760e55349542ddaf855c0fe60d782c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a6306c756d14d899623fc8b45e187ce" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46507 } } peers { permanent_uuid: "b3760e55349542ddaf855c0fe60d782c" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33957 } } peers { permanent_uuid: "9f3248e0f8d444edb82c65ff34e884a9" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34133 } }
I20260926 16:34:50.099721 25566 raft_consensus.cc:374] T b899f5bd8cb444b6b077d07050a01ebf P b3760e55349542ddaf855c0fe60d782c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:34:50.099781 25566 raft_consensus.cc:729] T b899f5bd8cb444b6b077d07050a01ebf P b3760e55349542ddaf855c0fe60d782c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3760e55349542ddaf855c0fe60d782c, State: Initialized, Role: FOLLOWER
I20260926 16:34:50.100260 25566 consensus_queue.cc:260] T b899f5bd8cb444b6b077d07050a01ebf P b3760e55349542ddaf855c0fe60d782c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a6306c756d14d899623fc8b45e187ce" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46507 } } peers { permanent_uuid: "b3760e55349542ddaf855c0fe60d782c" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33957 } } peers { permanent_uuid: "9f3248e0f8d444edb82c65ff34e884a9" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34133 } }
I20260926 16:34:50.100950 25547 heartbeater.cc:503] Master 127.11.8.254:46389 was elected leader, sending a full tablet report...
I20260926 16:34:50.101195 25566 ts_tablet_manager.cc:1428] T b899f5bd8cb444b6b077d07050a01ebf P b3760e55349542ddaf855c0fe60d782c: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260926 16:34:50.119669 25570 raft_consensus.cc:482] T b899f5bd8cb444b6b077d07050a01ebf P 9f3248e0f8d444edb82c65ff34e884a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:34:50.119864 25570 raft_consensus.cc:504] T b899f5bd8cb444b6b077d07050a01ebf P 9f3248e0f8d444edb82c65ff34e884a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a6306c756d14d899623fc8b45e187ce" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46507 } } peers { permanent_uuid: "b3760e55349542ddaf855c0fe60d782c" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33957 } } peers { permanent_uuid: "9f3248e0f8d444edb82c65ff34e884a9" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34133 } }
I20260926 16:34:50.121083 25570 leader_election.cc:288] T b899f5bd8cb444b6b077d07050a01ebf P 9f3248e0f8d444edb82c65ff34e884a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a6306c756d14d899623fc8b45e187ce (127.11.8.193:46507), b3760e55349542ddaf855c0fe60d782c (127.11.8.195:33957)
I20260926 16:34:50.125270 25415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:50.121376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:46507 (local address 127.11.8.194:47051)
0926 16:34:50.121486 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:50.121500 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:34:50.121630 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:34:50.121889 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:34:50.121894 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:34:50.121916 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:34:50.122040 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:50.122046 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.123045 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.123049 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:50.124221 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:50.124231 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.124942 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.124945 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:50.125025 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:50.125044 (+ 19us) client_negotiation.cc:769] Sending connection context
0926 16:34:50.125102 (+ 58us) client_negotiation.cc:240] Negotiation successful
0926 16:34:50.125156 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":5}
I20260926 16:34:50.125367 25559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:50.121526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:47051 (local address 127.11.8.193:46507)
0926 16:34:50.121672 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:50.121677 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:50.121689 (+ 12us) server_negotiation.cc:407] Connection header received
0926 16:34:50.121735 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:50.121738 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:50.121778 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:50.121848 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:34:50.122193 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.122948 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.124351 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.124835 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.124948 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:50.125000 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:50.125149 (+ 149us) server_negotiation.cc:299] Negotiation successful
0926 16:34:50.125221 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 16:34:50.125787 25573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:50.121376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:33957 (local address 127.11.8.194:37413)
0926 16:34:50.122116 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:50.122131 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:34:50.122232 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:34:50.122513 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:34:50.122516 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:34:50.122531 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:34:50.122657 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:50.122663 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.123497 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.123501 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:50.124542 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:50.124549 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.125373 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.125377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:50.125485 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:50.125520 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 16:34:50.125581 (+ 61us) client_negotiation.cc:240] Negotiation successful
0926 16:34:50.125641 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"mutex_wait_us":347,"thread_start_us":144,"threads_started":1}
I20260926 16:34:50.125809 25562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:50.121520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:37413 (local address 127.11.8.195:33957)
0926 16:34:50.121661 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:50.121667 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:50.122230 (+ 563us) server_negotiation.cc:407] Connection header received
0926 16:34:50.122348 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:50.122351 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:50.122404 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:50.122484 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:34:50.122773 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.123385 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.124650 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.125231 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.125342 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:50.125387 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:50.125613 (+ 226us) server_negotiation.cc:299] Negotiation successful
0926 16:34:50.125704 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 16:34:50.125947 25233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b899f5bd8cb444b6b077d07050a01ebf" candidate_uuid: "9f3248e0f8d444edb82c65ff34e884a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a6306c756d14d899623fc8b45e187ce" is_pre_election: true
I20260926 16:34:50.126307 25233 raft_consensus.cc:2454] T b899f5bd8cb444b6b077d07050a01ebf P 7a6306c756d14d899623fc8b45e187ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f3248e0f8d444edb82c65ff34e884a9 in term 0.
I20260926 16:34:50.126343 25501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b899f5bd8cb444b6b077d07050a01ebf" candidate_uuid: "9f3248e0f8d444edb82c65ff34e884a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3760e55349542ddaf855c0fe60d782c" is_pre_election: true
I20260926 16:34:50.126626 25501 raft_consensus.cc:2454] T b899f5bd8cb444b6b077d07050a01ebf P b3760e55349542ddaf855c0fe60d782c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f3248e0f8d444edb82c65ff34e884a9 in term 0.
I20260926 16:34:50.126814 25302 leader_election.cc:302] T b899f5bd8cb444b6b077d07050a01ebf P 9f3248e0f8d444edb82c65ff34e884a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a6306c756d14d899623fc8b45e187ce, 9f3248e0f8d444edb82c65ff34e884a9; no voters:
I20260926 16:34:50.127115 25570 raft_consensus.cc:2781] T b899f5bd8cb444b6b077d07050a01ebf P 9f3248e0f8d444edb82c65ff34e884a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:34:50.127202 25570 raft_consensus.cc:482] T b899f5bd8cb444b6b077d07050a01ebf P 9f3248e0f8d444edb82c65ff34e884a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:34:50.127234 25570 raft_consensus.cc:3037] T b899f5bd8cb444b6b077d07050a01ebf P 9f3248e0f8d444edb82c65ff34e884a9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:34:50.128350 25570 raft_consensus.cc:504] T b899f5bd8cb444b6b077d07050a01ebf P 9f3248e0f8d444edb82c65ff34e884a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a6306c756d14d899623fc8b45e187ce" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46507 } } peers { permanent_uuid: "b3760e55349542ddaf855c0fe60d782c" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33957 } } peers { permanent_uuid: "9f3248e0f8d444edb82c65ff34e884a9" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34133 } }
I20260926 16:34:50.128717 25570 leader_election.cc:288] T b899f5bd8cb444b6b077d07050a01ebf P 9f3248e0f8d444edb82c65ff34e884a9 [CANDIDATE]: Term 1 election: Requested vote from peers 7a6306c756d14d899623fc8b45e187ce (127.11.8.193:46507), b3760e55349542ddaf855c0fe60d782c (127.11.8.195:33957)
I20260926 16:34:50.129046 25501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b899f5bd8cb444b6b077d07050a01ebf" candidate_uuid: "9f3248e0f8d444edb82c65ff34e884a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3760e55349542ddaf855c0fe60d782c"
I20260926 16:34:50.129046 25233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b899f5bd8cb444b6b077d07050a01ebf" candidate_uuid: "9f3248e0f8d444edb82c65ff34e884a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a6306c756d14d899623fc8b45e187ce"
I20260926 16:34:50.129166 25501 raft_consensus.cc:3037] T b899f5bd8cb444b6b077d07050a01ebf P b3760e55349542ddaf855c0fe60d782c [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:34:50.129174 25233 raft_consensus.cc:3037] T b899f5bd8cb444b6b077d07050a01ebf P 7a6306c756d14d899623fc8b45e187ce [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:34:50.130448 25233 raft_consensus.cc:2454] T b899f5bd8cb444b6b077d07050a01ebf P 7a6306c756d14d899623fc8b45e187ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f3248e0f8d444edb82c65ff34e884a9 in term 1.
I20260926 16:34:50.130448 25501 raft_consensus.cc:2454] T b899f5bd8cb444b6b077d07050a01ebf P b3760e55349542ddaf855c0fe60d782c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f3248e0f8d444edb82c65ff34e884a9 in term 1.
I20260926 16:34:50.130729 25300 leader_election.cc:302] T b899f5bd8cb444b6b077d07050a01ebf P 9f3248e0f8d444edb82c65ff34e884a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f3248e0f8d444edb82c65ff34e884a9, b3760e55349542ddaf855c0fe60d782c; no voters:
I20260926 16:34:50.130919 25570 raft_consensus.cc:2781] T b899f5bd8cb444b6b077d07050a01ebf P 9f3248e0f8d444edb82c65ff34e884a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:34:50.131153 25570 raft_consensus.cc:686] T b899f5bd8cb444b6b077d07050a01ebf P 9f3248e0f8d444edb82c65ff34e884a9 [term 1 LEADER]: Becoming Leader. State: Replica: 9f3248e0f8d444edb82c65ff34e884a9, State: Running, Role: LEADER
I20260926 16:34:50.131474 25570 consensus_queue.cc:237] T b899f5bd8cb444b6b077d07050a01ebf P 9f3248e0f8d444edb82c65ff34e884a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a6306c756d14d899623fc8b45e187ce" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46507 } } peers { permanent_uuid: "b3760e55349542ddaf855c0fe60d782c" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33957 } } peers { permanent_uuid: "9f3248e0f8d444edb82c65ff34e884a9" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34133 } }
I20260926 16:34:50.134629 25088 catalog_manager.cc:5495] T b899f5bd8cb444b6b077d07050a01ebf P 9f3248e0f8d444edb82c65ff34e884a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f3248e0f8d444edb82c65ff34e884a9 (127.11.8.194). New cstate: current_term: 1 leader_uuid: "9f3248e0f8d444edb82c65ff34e884a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a6306c756d14d899623fc8b45e187ce" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3760e55349542ddaf855c0fe60d782c" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f3248e0f8d444edb82c65ff34e884a9" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34133 } health_report { overall_health: HEALTHY } } }
I20260926 16:34:50.146706 25563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:50.143362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32858 (local address 127.11.8.194:34133)
0926 16:34:50.143498 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:50.143502 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:50.143515 (+ 13us) server_negotiation.cc:407] Connection header received
0926 16:34:50.143621 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:50.143624 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:50.143674 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:50.143761 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:34:50.143964 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.144485 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.145276 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.145490 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.145575 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:50.145852 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:34:50.146366 (+ 514us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:34:50.146466 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:50.146524 (+ 58us) server_negotiation.cc:299] Negotiation successful
0926 16:34:50.146577 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 16:34:50.152468 25233 raft_consensus.cc:1261] T b899f5bd8cb444b6b077d07050a01ebf P 7a6306c756d14d899623fc8b45e187ce [term 1 FOLLOWER]: Refusing update from remote peer 9f3248e0f8d444edb82c65ff34e884a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:34:50.152468 25501 raft_consensus.cc:1261] T b899f5bd8cb444b6b077d07050a01ebf P b3760e55349542ddaf855c0fe60d782c [term 1 FOLLOWER]: Refusing update from remote peer 9f3248e0f8d444edb82c65ff34e884a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:34:50.153275 25570 consensus_queue.cc:1035] T b899f5bd8cb444b6b077d07050a01ebf P 9f3248e0f8d444edb82c65ff34e884a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a6306c756d14d899623fc8b45e187ce" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:34:50.153534 25574 consensus_queue.cc:1035] T b899f5bd8cb444b6b077d07050a01ebf P 9f3248e0f8d444edb82c65ff34e884a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3760e55349542ddaf855c0fe60d782c" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:34:50.160475 25578 mvcc.cc:205] Tried to move back new op lower bound from 7333644247657283584 to 7333644247581298688. Current Snapshot: MvccSnapshot[applied={T|T < 7333644247657283584}]
I20260926 16:34:50.162400 25579 mvcc.cc:205] Tried to move back new op lower bound from 7333644247657283584 to 7333644247581298688. Current Snapshot: MvccSnapshot[applied={T|T < 7333644247657283584}]
I20260926 16:34:50.169983 25087 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47840:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:34:50.170123 25087 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47840:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:34:50.172416 25087 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158: Sending DeleteTablet for 3 replicas of tablet b899f5bd8cb444b6b077d07050a01ebf
I20260926 16:34:50.173171 25481 tablet_service.cc:1514] Processing DeleteTablet for tablet b899f5bd8cb444b6b077d07050a01ebf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:34:50 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:34368
I20260926 16:34:50.173148 25213 tablet_service.cc:1514] Processing DeleteTablet for tablet b899f5bd8cb444b6b077d07050a01ebf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:34:50 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:44802
I20260926 16:34:50.173316 25347 tablet_service.cc:1514] Processing DeleteTablet for tablet b899f5bd8cb444b6b077d07050a01ebf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:34:50 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:32850
I20260926 16:34:50.173671 25587 tablet_replica.cc:330] T b899f5bd8cb444b6b077d07050a01ebf P 7a6306c756d14d899623fc8b45e187ce: stopping tablet replica
I20260926 16:34:50.173877 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 25150
I20260926 16:34:50.177734 25588 tablet_replica.cc:330] T b899f5bd8cb444b6b077d07050a01ebf P b3760e55349542ddaf855c0fe60d782c: stopping tablet replica
I20260926 16:34:50.178018 25589 tablet_replica.cc:330] T b899f5bd8cb444b6b077d07050a01ebf P 9f3248e0f8d444edb82c65ff34e884a9: stopping tablet replica
I20260926 16:34:50.178053 25588 raft_consensus.cc:2229] T b899f5bd8cb444b6b077d07050a01ebf P b3760e55349542ddaf855c0fe60d782c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:34:50.178381 25588 raft_consensus.cc:2258] T b899f5bd8cb444b6b077d07050a01ebf P b3760e55349542ddaf855c0fe60d782c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:34:50.178526 25589 raft_consensus.cc:2229] T b899f5bd8cb444b6b077d07050a01ebf P 9f3248e0f8d444edb82c65ff34e884a9 [term 1 LEADER]: Raft consensus shutting down.
I20260926 16:34:50.178947 25589 raft_consensus.cc:2258] T b899f5bd8cb444b6b077d07050a01ebf P 9f3248e0f8d444edb82c65ff34e884a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:34:50.180333 25588 ts_tablet_manager.cc:1905] T b899f5bd8cb444b6b077d07050a01ebf P b3760e55349542ddaf855c0fe60d782c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:34:50.180333 25589 ts_tablet_manager.cc:1905] T b899f5bd8cb444b6b077d07050a01ebf P 9f3248e0f8d444edb82c65ff34e884a9: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 16:34:50.182020 25074 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.8.193:46507 (error 108)
I20260926 16:34:50.182807 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 25284
I20260926 16:34:50.183825 25588 ts_tablet_manager.cc:1918] T b899f5bd8cb444b6b077d07050a01ebf P b3760e55349542ddaf855c0fe60d782c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:34:50.183943 25588 log.cc:1196] T b899f5bd8cb444b6b077d07050a01ebf P b3760e55349542ddaf855c0fe60d782c: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/wal/wals/b899f5bd8cb444b6b077d07050a01ebf
I20260926 16:34:50.184226 25588 ts_tablet_manager.cc:1939] T b899f5bd8cb444b6b077d07050a01ebf P b3760e55349542ddaf855c0fe60d782c: Deleting consensus metadata
I20260926 16:34:50.185281 25072 catalog_manager.cc:4841] TS b3760e55349542ddaf855c0fe60d782c (127.11.8.195:33957): tablet b899f5bd8cb444b6b077d07050a01ebf (table test-table [id=5c76558d1226418da10f82508e2b3370]) successfully deleted
I20260926 16:34:50.185859 25558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:50.185603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:46507 (local address 127.0.0.1:44814)
0926 16:34:50.185722 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:50.185790 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.193:46507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260926 16:34:50.186086 25074 catalog_manager.cc:4584] TS 7a6306c756d14d899623fc8b45e187ce (127.11.8.193:46507): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b899f5bd8cb444b6b077d07050a01ebf: Network error: Client connection negotiation failed: client connection to 127.11.8.193:46507: connect: Connection refused (error 111)
I20260926 16:34:50.188544 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 25418
I20260926 16:34:50.190362 25558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:50.188729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:34133 (local address 127.0.0.1:32862)
0926 16:34:50.190150 (+ 1421us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:50.190195 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.194:34133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1357}
I20260926 16:34:50.194061 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 25056
I20260926 16:34:50.200205 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:46389
--webserver_interface=127.11.8.254
--webserver_port=34519
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:39665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:46389
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:34:50.307478 25591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:34:50.307714 25591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:34:50.307751 25591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:34:50.311666 25591 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:34:50.311714 25591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:34:50.311733 25591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:34:50.311749 25591 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:34:50.311764 25591 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:34:50.316874 25591 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:39665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:46389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:46389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=34519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:34:50.317823 25591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:34:50.318799 25591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:34:50.331331 25600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:34:50.331503 25591 server_base.cc:1034] running on GCE node
W20260926 16:34:50.331279 25597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:34:50.331277 25598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:34:50.332304 25591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:34:50.333328 25591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:34:50.334517 25591 hybrid_clock.cc:648] HybridClock initialized: now 1790440490334498 us; error 29 us; skew 500 ppm
Sep 26 16:34:50 dist-test-slave-2zvq krb5kdc[25043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440490, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:34:50.337252 25591 init.cc:377] Logged in from keytab as kudu/127.11.8.254@KRBTEST.COM (short username kudu)
I20260926 16:34:50.338119 25591 webserver.cc:460] Webserver started at http://127.11.8.254:34519/ using document root <none> and password file <none>
I20260926 16:34:50.338737 25591 fs_manager.cc:359] Metadata directory not provided
I20260926 16:34:50.338838 25591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:34:50.342675 25591 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260926 16:34:50.344802 25607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:50.346051 25591 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:34:50.346192 25591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "620a6df881a040c7b5d5266f54fb2158"
format_stamp: "Formatted at 2026-09-26 16:34:49 on dist-test-slave-2zvq"
server_key: "7d6281c966055fc1ecb90d7a8cb0a8b1"
server_key_iv: "d8240f03fa1092e08882b6407ccad068"
server_key_version: "encryptionkey@0"
I20260926 16:34:50.346885 25591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:34:50.363157 25591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:34:50.365830 25591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:34:50.366025 25591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:34:50.372841 25591 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:46389
I20260926 16:34:50.373123 25659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:46389 every 8 connection(s)
I20260926 16:34:50.373910 25591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:34:50.375903 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 25591
I20260926 16:34:50.376382 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.193:46507
--local_ip_for_outbound_sockets=127.11.8.193
--tserver_master_addrs=127.11.8.254:46389
--webserver_port=46449
--webserver_interface=127.11.8.193
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:39665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 16:34:50.379082 25660 sys_catalog.cc:263] Verifying existing consensus state
I20260926 16:34:50.380729 25660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158: Bootstrap starting.
I20260926 16:34:50.398315 25660 log.cc:824] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158: Log is configured to *not* fsync() on all Append() calls
I20260926 16:34:50.406816 25660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 16:34:50.407204 25660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158: Bootstrap complete.
I20260926 16:34:50.411441 25660 raft_consensus.cc:348] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "620a6df881a040c7b5d5266f54fb2158" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 46389 } }
I20260926 16:34:50.411823 25660 raft_consensus.cc:729] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 620a6df881a040c7b5d5266f54fb2158, State: Initialized, Role: FOLLOWER
I20260926 16:34:50.412302 25660 consensus_queue.cc:260] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "620a6df881a040c7b5d5266f54fb2158" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 46389 } }
I20260926 16:34:50.412416 25660 raft_consensus.cc:388] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:34:50.412508 25660 raft_consensus.cc:482] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:34:50.412627 25660 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [term 1 FOLLOWER]: Advancing to term 2
I20260926 16:34:50.414534 25660 raft_consensus.cc:504] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "620a6df881a040c7b5d5266f54fb2158" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 46389 } }
I20260926 16:34:50.414858 25660 leader_election.cc:302] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 620a6df881a040c7b5d5266f54fb2158; no voters:
I20260926 16:34:50.415201 25660 leader_election.cc:288] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 16:34:50.415288 25664 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 16:34:50.415482 25664 raft_consensus.cc:686] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [term 2 LEADER]: Becoming Leader. State: Replica: 620a6df881a040c7b5d5266f54fb2158, State: Running, Role: LEADER
I20260926 16:34:50.415772 25664 consensus_queue.cc:237] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "620a6df881a040c7b5d5266f54fb2158" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 46389 } }
I20260926 16:34:50.416474 25660 sys_catalog.cc:564] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:34:50.416689 25664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 620a6df881a040c7b5d5266f54fb2158. Latest consensus state: current_term: 2 leader_uuid: "620a6df881a040c7b5d5266f54fb2158" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "620a6df881a040c7b5d5266f54fb2158" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 46389 } } }
I20260926 16:34:50.416793 25664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [sys.catalog]: This master's current role is: LEADER
I20260926 16:34:50.417146 25668 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:34:50.417241 25665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "620a6df881a040c7b5d5266f54fb2158" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "620a6df881a040c7b5d5266f54fb2158" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 46389 } } }
I20260926 16:34:50.417413 25665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [sys.catalog]: This master's current role is: LEADER
I20260926 16:34:50.421873 25668 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:34:50.422513 25668 catalog_manager.cc:1259] Loaded cluster ID: bb4e410281e241999082312ec85add52
I20260926 16:34:50.422581 25668 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:34:50.423980 25668 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260926 16:34:50.429155 25668 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260926 16:34:50.434439 25668 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:34:50.435385 25668 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158: Loaded TSK: 0
I20260926 16:34:50.436573 25668 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 16:34:50.489271 25662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:34:50.489550 25662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:34:50.489625 25662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:34:50.493613 25662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:34:50.493706 25662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:34:50.493796 25662 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.193
I20260926 16:34:50.499063 25662 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:39665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.193:46507
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.193
--webserver_port=46449
--tserver_master_addrs=127.11.8.254:46389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.193
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:34:50.500134 25662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:34:50.501204 25662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:34:50.511807 25688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:34:50.512013 25662 server_base.cc:1034] running on GCE node
W20260926 16:34:50.511806 25690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:34:50.511807 25687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:34:50.512650 25662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:34:50.513391 25662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:34:50.514573 25662 hybrid_clock.cc:648] HybridClock initialized: now 1790440490514540 us; error 41 us; skew 500 ppm
Sep 26 16:34:50 dist-test-slave-2zvq krb5kdc[25043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440490, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:34:50.517963 25662 init.cc:377] Logged in from keytab as kudu/127.11.8.193@KRBTEST.COM (short username kudu)
I20260926 16:34:50.518877 25662 webserver.cc:460] Webserver started at http://127.11.8.193:46449/ using document root <none> and password file <none>
I20260926 16:34:50.519431 25662 fs_manager.cc:359] Metadata directory not provided
I20260926 16:34:50.519503 25662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:34:50.523289 25662 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260926 16:34:50.525465 25697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:50.526784 25662 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:34:50.526918 25662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/wal
uuid: "7a6306c756d14d899623fc8b45e187ce"
format_stamp: "Formatted at 2026-09-26 16:34:49 on dist-test-slave-2zvq"
server_key: "0c4b5b14b95a43edce8654daee6cf0fa"
server_key_iv: "e8d2f45270d6fa44001e5e8f6d12009a"
server_key_version: "encryptionkey@0"
I20260926 16:34:50.527632 25662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:34:50.545207 25662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:34:50.548585 25662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:34:50.548796 25662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:34:50.549377 25662 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:34:50.550580 25704 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 16:34:50.554814 25662 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 16:34:50.554876 25662 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260926 16:34:50.554922 25662 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 16:34:50.556516 25662 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 16:34:50.556571 25662 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260926 16:34:50.556828 25704 tablet_bootstrap.cc:492] T b899f5bd8cb444b6b077d07050a01ebf P 7a6306c756d14d899623fc8b45e187ce: Bootstrap starting.
I20260926 16:34:50.567101 25662 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.193:46507
I20260926 16:34:50.567301 25811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.193:46507 every 8 connection(s)
I20260926 16:34:50.568317 25662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
I20260926 16:34:50.572965 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 25662
I20260926 16:34:50.573446 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.194:34133
--local_ip_for_outbound_sockets=127.11.8.194
--tserver_master_addrs=127.11.8.254:46389
--webserver_port=45385
--webserver_interface=127.11.8.194
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:39665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 16:34:50.578410 25704 log.cc:824] T b899f5bd8cb444b6b077d07050a01ebf P 7a6306c756d14d899623fc8b45e187ce: Log is configured to *not* fsync() on all Append() calls
Sep 26 16:34:50 dist-test-slave-2zvq krb5kdc[25043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440490, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:34:50.586447 25704 tablet_bootstrap.cc:492] T b899f5bd8cb444b6b077d07050a01ebf P 7a6306c756d14d899623fc8b45e187ce: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 16:34:50.586846 25704 tablet_bootstrap.cc:492] T b899f5bd8cb444b6b077d07050a01ebf P 7a6306c756d14d899623fc8b45e187ce: Bootstrap complete.
I20260926 16:34:50.588207 25704 ts_tablet_manager.cc:1397] T b899f5bd8cb444b6b077d07050a01ebf P 7a6306c756d14d899623fc8b45e187ce: Time spent bootstrapping tablet: real 0.032s user 0.024s sys 0.004s
I20260926 16:34:50.589031 25814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:50.570446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:34137 (local address 127.11.8.254:46389)
0926 16:34:50.570852 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:50.570870 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:50.571276 (+ 406us) server_negotiation.cc:407] Connection header received
0926 16:34:50.572773 (+ 1497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:50.572791 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:50.573239 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:50.573618 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:34:50.575434 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.577057 (+ 1623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.578827 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.579670 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.579882 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:50.582994 (+ 3112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:34:50.583003 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:34:50.583024 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:34:50.583067 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:34:50.585448 (+ 2381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:50.586078 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:50.586084 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:50.586090 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:50.586241 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:50.586663 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:50.586666 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:50.586668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:50.587119 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:34:50.587246 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:50.587910 (+ 664us) server_negotiation.cc:299] Negotiation successful
0926 16:34:50.588202 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":141,"threads_started":1}
I20260926 16:34:50.589048 25815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:50.570437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.193:34137)
0926 16:34:50.571085 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:50.571151 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0926 16:34:50.572336 (+ 1185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:34:50.573940 (+ 1604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:34:50.573949 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:34:50.574457 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:34:50.575229 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:50.575257 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.577229 (+ 1972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.577233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:50.578554 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:50.578566 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.579840 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.579844 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:50.580024 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:50.580753 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:34:50.580773 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:34:50.582747 (+ 1974us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:34:50.585579 (+ 2832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:50.585585 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:50.585597 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:50.585943 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:50.586386 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:50.586389 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:50.586391 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:50.586565 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:50.587243 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:34:50.587252 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:34:50.587552 (+ 300us) client_negotiation.cc:769] Sending connection context
0926 16:34:50.587753 (+ 201us) client_negotiation.cc:240] Negotiation successful
0926 16:34:50.588107 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":156,"threads_started":1}
I20260926 16:34:50.591081 25704 raft_consensus.cc:348] T b899f5bd8cb444b6b077d07050a01ebf P 7a6306c756d14d899623fc8b45e187ce [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a6306c756d14d899623fc8b45e187ce" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46507 } } peers { permanent_uuid: "b3760e55349542ddaf855c0fe60d782c" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33957 } } peers { permanent_uuid: "9f3248e0f8d444edb82c65ff34e884a9" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34133 } }
I20260926 16:34:50.591496 25704 raft_consensus.cc:729] T b899f5bd8cb444b6b077d07050a01ebf P 7a6306c756d14d899623fc8b45e187ce [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a6306c756d14d899623fc8b45e187ce, State: Initialized, Role: FOLLOWER
I20260926 16:34:50.591997 25704 consensus_queue.cc:260] T b899f5bd8cb444b6b077d07050a01ebf P 7a6306c756d14d899623fc8b45e187ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a6306c756d14d899623fc8b45e187ce" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46507 } } peers { permanent_uuid: "b3760e55349542ddaf855c0fe60d782c" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33957 } } peers { permanent_uuid: "9f3248e0f8d444edb82c65ff34e884a9" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34133 } }
I20260926 16:34:50.592291 25812 heartbeater.cc:347] Connected to a master server at 127.11.8.254:46389
I20260926 16:34:50.592522 25812 heartbeater.cc:464] Registering TS with master...
I20260926 16:34:50.592875 25704 ts_tablet_manager.cc:1428] T b899f5bd8cb444b6b077d07050a01ebf P 7a6306c756d14d899623fc8b45e187ce: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260926 16:34:50.593057 25812 heartbeater.cc:511] Master 127.11.8.254:46389 requested a full tablet report, sending...
I20260926 16:34:50.595624 25624 ts_manager.cc:194] Registered new tserver with Master: 7a6306c756d14d899623fc8b45e187ce (127.11.8.193:46507)
I20260926 16:34:50.598954 25624 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.193@KRBTEST.COM'} at 127.11.8.193:34137
I20260926 16:34:50.601567 25812 heartbeater.cc:503] Master 127.11.8.254:46389 was elected leader, sending a full tablet report...
Sep 26 16:34:50 dist-test-slave-2zvq krb5kdc[25043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440490, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for kudu/127.11.8.193@KRBTEST.COM
I20260926 16:34:50.609292 25819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:50.598379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:46507 (local address 127.0.0.1:44822)
0926 16:34:50.598954 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:50.598997 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 16:34:50.599151 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:34:50.600256 (+ 1105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:34:50.600262 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:34:50.600616 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:34:50.600716 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:50.600723 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.601654 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.601660 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:50.602452 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:50.602462 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.603206 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.603212 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:50.603304 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:50.603871 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:34:50.603882 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:34:50.605441 (+ 1559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:34:50.607518 (+ 2077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:50.607525 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:50.607529 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:50.607777 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:50.608097 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:50.608102 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:50.608104 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:50.608197 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:50.608661 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:34:50.608667 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:34:50.608783 (+ 116us) client_negotiation.cc:769] Sending connection context
0926 16:34:50.608918 (+ 135us) client_negotiation.cc:240] Negotiation successful
0926 16:34:50.609113 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":111,"threads_started":1}
I20260926 16:34:50.609341 25820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:50.598463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44822 (local address 127.11.8.193:46507)
0926 16:34:50.599826 (+ 1363us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:50.599838 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:50.599858 (+ 20us) server_negotiation.cc:407] Connection header received
0926 16:34:50.599919 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:50.599923 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:50.600109 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:50.600257 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:34:50.600843 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.601531 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.602578 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.603057 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.603162 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:50.605575 (+ 2413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:34:50.605584 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:34:50.605596 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:34:50.605621 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:34:50.607405 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:50.607905 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:50.607911 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:50.607913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:50.608000 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:50.608293 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:50.608298 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:50.608300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:50.608536 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:34:50.608680 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:50.609006 (+ 326us) server_negotiation.cc:299] Negotiation successful
0926 16:34:50.609183 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1271,"thread_start_us":118,"threads_started":1}
I20260926 16:34:50.610219 25746 tablet_service.cc:1514] Processing DeleteTablet for tablet b899f5bd8cb444b6b077d07050a01ebf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:34:50 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:44822
I20260926 16:34:50.611147 25821 tablet_replica.cc:330] stopping tablet replica
I20260926 16:34:50.611467 25821 raft_consensus.cc:2229] T b899f5bd8cb444b6b077d07050a01ebf P 7a6306c756d14d899623fc8b45e187ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:34:50.611726 25821 raft_consensus.cc:2258] T b899f5bd8cb444b6b077d07050a01ebf P 7a6306c756d14d899623fc8b45e187ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:34:50.612835 25821 ts_tablet_manager.cc:1905] T b899f5bd8cb444b6b077d07050a01ebf P 7a6306c756d14d899623fc8b45e187ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:34:50.617362 25821 ts_tablet_manager.cc:1918] T b899f5bd8cb444b6b077d07050a01ebf P 7a6306c756d14d899623fc8b45e187ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:34:50.617457 25821 log.cc:1196] T b899f5bd8cb444b6b077d07050a01ebf P 7a6306c756d14d899623fc8b45e187ce: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/wal/wals/b899f5bd8cb444b6b077d07050a01ebf
I20260926 16:34:50.617730 25821 ts_tablet_manager.cc:1939] T b899f5bd8cb444b6b077d07050a01ebf P 7a6306c756d14d899623fc8b45e187ce: Deleting consensus metadata
I20260926 16:34:50.619720 25610 catalog_manager.cc:4841] TS 7a6306c756d14d899623fc8b45e187ce (127.11.8.193:46507): tablet b899f5bd8cb444b6b077d07050a01ebf (table test-table [id=5c76558d1226418da10f82508e2b3370]) successfully deleted
W20260926 16:34:50.691890 25816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:34:50.692123 25816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:34:50.692163 25816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:34:50.696111 25816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:34:50.696159 25816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:34:50.696254 25816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.194
I20260926 16:34:50.701256 25816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:39665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.194:34133
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.194
--webserver_port=45385
--tserver_master_addrs=127.11.8.254:46389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.194
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:34:50.702188 25816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:34:50.703183 25816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:34:50.712740 25829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:34:50.712911 25816 server_base.cc:1034] running on GCE node
W20260926 16:34:50.712776 25827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:34:50.712736 25826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:34:50.713425 25816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:34:50.714077 25816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:34:50.715273 25816 hybrid_clock.cc:648] HybridClock initialized: now 1790440490715267 us; error 50 us; skew 500 ppm
Sep 26 16:34:50 dist-test-slave-2zvq krb5kdc[25043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440490, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:34:50.718322 25816 init.cc:377] Logged in from keytab as kudu/127.11.8.194@KRBTEST.COM (short username kudu)
I20260926 16:34:50.719210 25816 webserver.cc:460] Webserver started at http://127.11.8.194:45385/ using document root <none> and password file <none>
I20260926 16:34:50.719801 25816 fs_manager.cc:359] Metadata directory not provided
I20260926 16:34:50.719904 25816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:34:50.723523 25816 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:34:50.725627 25836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:50.726861 25816 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:34:50.727000 25816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/wal
uuid: "9f3248e0f8d444edb82c65ff34e884a9"
format_stamp: "Formatted at 2026-09-26 16:34:49 on dist-test-slave-2zvq"
server_key: "cc4d9911c1a7a00ea1f1d467d1aa0c56"
server_key_iv: "ec7e67e6371902dcb3a82eeb64c2d647"
server_key_version: "encryptionkey@0"
I20260926 16:34:50.727543 25816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:34:50.744336 25816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:34:50.747684 25816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:34:50.747833 25816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:34:50.748356 25816 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:34:50.749372 25843 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260926 16:34:50.753487 25843 ts_tablet_manager.cc:1852] T b899f5bd8cb444b6b077d07050a01ebf P 9f3248e0f8d444edb82c65ff34e884a9: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260926 16:34:50.753881 25843 ts_tablet_manager.cc:1905] T b899f5bd8cb444b6b077d07050a01ebf P 9f3248e0f8d444edb82c65ff34e884a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:34:50.757781 25843 ts_tablet_manager.cc:1918] T b899f5bd8cb444b6b077d07050a01ebf P 9f3248e0f8d444edb82c65ff34e884a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:34:50.757879 25843 log.cc:1196] T b899f5bd8cb444b6b077d07050a01ebf P 9f3248e0f8d444edb82c65ff34e884a9: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/wal/wals/b899f5bd8cb444b6b077d07050a01ebf
I20260926 16:34:50.758343 25843 ts_tablet_manager.cc:1939] T b899f5bd8cb444b6b077d07050a01ebf P 9f3248e0f8d444edb82c65ff34e884a9: Deleting consensus metadata
I20260926 16:34:50.758670 25816 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260926 16:34:50.758735 25816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.010s user 0.000s sys 0.000s
I20260926 16:34:50.758824 25816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:34:50.758863 25816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:50.767652 25816 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.194:34133
I20260926 16:34:50.767702 25950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.194:34133 every 8 connection(s)
I20260926 16:34:50.768476 25816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
I20260926 16:34:50.770421 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 25816
I20260926 16:34:50.770798 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.195:33957
--local_ip_for_outbound_sockets=127.11.8.195
--tserver_master_addrs=127.11.8.254:46389
--webserver_port=37475
--webserver_interface=127.11.8.195
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:39665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 16:34:50 dist-test-slave-2zvq krb5kdc[25043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440490, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:34:50.782948 25814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:50.770239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:48617 (local address 127.11.8.254:46389)
0926 16:34:50.770436 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:50.770442 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:50.771388 (+ 946us) server_negotiation.cc:407] Connection header received
0926 16:34:50.772313 (+ 925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:50.772319 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:50.772384 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:50.772494 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:34:50.773796 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.774393 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.776226 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.776445 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.776526 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:50.779437 (+ 2911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:34:50.779440 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:34:50.779442 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:34:50.779465 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:34:50.780909 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:50.781407 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:50.781411 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:50.781413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:50.781495 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:50.781821 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:50.781826 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:50.781828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:50.782003 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:34:50.782109 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:50.782638 (+ 529us) server_negotiation.cc:299] Negotiation successful
0926 16:34:50.782779 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260926 16:34:50.783694 25953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:50.770702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.194:48617)
0926 16:34:50.771225 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:50.771276 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 16:34:50.772106 (+ 830us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:34:50.772656 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:34:50.772663 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:34:50.772993 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:34:50.773636 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:50.773659 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.774520 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.774525 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:50.776094 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:50.776107 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.776607 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.776614 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:50.776849 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:50.777521 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:34:50.777546 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:34:50.779265 (+ 1719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:34:50.781025 (+ 1760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:50.781031 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:50.781044 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:50.781305 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:50.781587 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:50.781592 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:50.781594 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:50.781726 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:50.782105 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:34:50.782112 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:34:50.782408 (+ 296us) client_negotiation.cc:769] Sending connection context
0926 16:34:50.782584 (+ 176us) client_negotiation.cc:240] Negotiation successful
0926 16:34:50.782883 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":130,"threads_started":1}
I20260926 16:34:50.784861 25951 heartbeater.cc:347] Connected to a master server at 127.11.8.254:46389
I20260926 16:34:50.785100 25951 heartbeater.cc:464] Registering TS with master...
I20260926 16:34:50.785564 25951 heartbeater.cc:511] Master 127.11.8.254:46389 requested a full tablet report, sending...
I20260926 16:34:50.786703 25624 ts_manager.cc:194] Registered new tserver with Master: 9f3248e0f8d444edb82c65ff34e884a9 (127.11.8.194:34133)
I20260926 16:34:50.787497 25624 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.194@KRBTEST.COM'} at 127.11.8.194:48617
W20260926 16:34:50.880556 25954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:34:50.880792 25954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:34:50.880834 25954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:34:50.884660 25954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:34:50.884708 25954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:34:50.884763 25954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.195
I20260926 16:34:50.889679 25954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:39665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.195:33957
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.195
--webserver_port=37475
--tserver_master_addrs=127.11.8.254:46389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.195
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:34:50.890579 25954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:34:50.891472 25954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:34:50.900689 25960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:34:50.900655 25959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:34:50.900808 25954 server_base.cc:1034] running on GCE node
W20260926 16:34:50.900655 25962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:34:50.901325 25954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:34:50.901917 25954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:34:50.903072 25954 hybrid_clock.cc:648] HybridClock initialized: now 1790440490902954 us; error 131 us; skew 500 ppm
Sep 26 16:34:50 dist-test-slave-2zvq krb5kdc[25043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440490, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:34:50.905716 25954 init.cc:377] Logged in from keytab as kudu/127.11.8.195@KRBTEST.COM (short username kudu)
I20260926 16:34:50.906620 25954 webserver.cc:460] Webserver started at http://127.11.8.195:37475/ using document root <none> and password file <none>
I20260926 16:34:50.907158 25954 fs_manager.cc:359] Metadata directory not provided
I20260926 16:34:50.907281 25954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:34:50.911298 25954 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 16:34:50.913352 25969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:50.914693 25954 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:34:50.914827 25954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/wal
uuid: "b3760e55349542ddaf855c0fe60d782c"
format_stamp: "Formatted at 2026-09-26 16:34:49 on dist-test-slave-2zvq"
server_key: "9c82336090ac85ab12e89d56946fbad8"
server_key_iv: "2b166f5ad0f0fc921589dbc5bb521f90"
server_key_version: "encryptionkey@0"
I20260926 16:34:50.915362 25954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:34:50.940977 25954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:34:50.944375 25954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:34:50.944574 25954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:34:50.945163 25954 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:34:50.945953 25954 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:34:50.946023 25954 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:50.946099 25954 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:34:50.946151 25954 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:50.955224 25954 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.195:33957
I20260926 16:34:50.955250 26082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.195:33957 every 8 connection(s)
I20260926 16:34:50.956173 25954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
I20260926 16:34:50.957348 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 25954
Sep 26 16:34:50 dist-test-slave-2zvq krb5kdc[25043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440490, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:34:50.967213 26086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:50.958149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47846 (local address 127.11.8.254:46389)
0926 16:34:50.958350 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:50.958354 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:50.958547 (+ 193us) server_negotiation.cc:407] Connection header received
0926 16:34:50.958751 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:50.958754 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:50.958802 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:50.958894 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:34:50.960151 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.960721 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.961921 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.962229 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.962385 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:50.963719 (+ 1334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:34:50.963723 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:34:50.963726 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:34:50.963750 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:34:50.965402 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:50.965828 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:50.965833 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:50.965835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:50.965925 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:50.966229 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:50.966232 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:50.966233 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:50.966455 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:34:50.966619 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:50.966859 (+ 240us) server_negotiation.cc:299] Negotiation successful
0926 16:34:50.967017 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":85,"threads_started":1}
I20260926 16:34:50.970511 25814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:50.958053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:47609 (local address 127.11.8.254:46389)
0926 16:34:50.958190 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:50.958195 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:50.959134 (+ 939us) server_negotiation.cc:407] Connection header received
0926 16:34:50.960712 (+ 1578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:50.960715 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:50.960764 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:50.960842 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:34:50.962229 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.962836 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.963905 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.964208 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.964313 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:50.966704 (+ 2391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:34:50.966707 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:34:50.966709 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:34:50.966734 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:34:50.968403 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:50.968959 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:50.968962 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:50.968963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:50.969077 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:50.969544 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:50.969546 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:50.969548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:50.969733 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:34:50.969811 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:50.970315 (+ 504us) server_negotiation.cc:299] Negotiation successful
0926 16:34:50.970416 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 16:34:50.971306 26087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:50.958491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.195:47609)
0926 16:34:50.958938 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:50.958987 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 16:34:50.960518 (+ 1531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:34:50.961046 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:34:50.961055 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:34:50.961500 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:34:50.962084 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:50.962105 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.962939 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.962943 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:50.963795 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:50.963803 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.964351 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.964354 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:50.964487 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:50.965032 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:34:50.965047 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:34:50.966536 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:34:50.968511 (+ 1975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:50.968516 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:50.968530 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:50.968845 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:50.969293 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:50.969296 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:50.969297 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:50.969428 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:50.969852 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:34:50.969857 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:34:50.970065 (+ 208us) client_negotiation.cc:769] Sending connection context
0926 16:34:50.970221 (+ 156us) client_negotiation.cc:240] Negotiation successful
0926 16:34:50.970553 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":129,"threads_started":1}
I20260926 16:34:50.972354 26083 heartbeater.cc:347] Connected to a master server at 127.11.8.254:46389
I20260926 16:34:50.972582 26083 heartbeater.cc:464] Registering TS with master...
I20260926 16:34:50.973029 26083 heartbeater.cc:511] Master 127.11.8.254:46389 requested a full tablet report, sending...
I20260926 16:34:50.973986 25624 ts_manager.cc:194] Registered new tserver with Master: b3760e55349542ddaf855c0fe60d782c (127.11.8.195:33957)
I20260926 16:34:50.974560 25624 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.195@KRBTEST.COM'} at 127.11.8.195:47609
I20260926 16:34:50.981062 11299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 16:34:50.984987 25814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:50.981641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47858 (local address 127.11.8.254:46389)
0926 16:34:50.981744 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:50.981748 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:50.982034 (+ 286us) server_negotiation.cc:407] Connection header received
0926 16:34:50.982103 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:50.982105 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:50.982187 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:50.982327 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:34:50.982538 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.983043 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.983845 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:50.984045 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:50.984145 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:50.984364 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:34:50.984700 (+ 336us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:34:50.984787 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:50.984856 (+ 69us) server_negotiation.cc:299] Negotiation successful
0926 16:34:50.984898 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 16:34:50.985625 25624 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:47858:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:34:50.988840 25624 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:34:50.999349 25746 tablet_service.cc:1467] Processing CreateTablet for tablet 72d0a9d902804b2c9f418b45c3196008 (DEFAULT_TABLE table=test-table [id=fd2bf7b293834f9bbbc64ee5580cd74d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:34:51.000044 25746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72d0a9d902804b2c9f418b45c3196008. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:34:51.003237 25704 tablet_bootstrap.cc:492] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce: Bootstrap starting.
I20260926 16:34:51.004107 25704 tablet_bootstrap.cc:654] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce: Neither blocks nor log segments found. Creating new log.
I20260926 16:34:51.004984 25819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:50.997708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:33957 (local address 127.0.0.1:34384)
0926 16:34:50.997845 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:50.997860 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:34:50.997935 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:34:51.000522 (+ 2587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:34:51.000524 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:34:51.000582 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:34:51.000687 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:51.000697 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:51.002461 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:51.002466 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:51.003460 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:51.003471 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:51.004534 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:51.004537 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:51.004652 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:51.004704 (+ 52us) client_negotiation.cc:769] Sending connection context
0926 16:34:51.004760 (+ 56us) client_negotiation.cc:240] Negotiation successful
0926 16:34:51.004836 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260926 16:34:51.005577 26092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:50.998113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34384 (local address 127.11.8.195:33957)
0926 16:34:50.999986 (+ 1873us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:50.999996 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:51.000020 (+ 24us) server_negotiation.cc:407] Connection header received
0926 16:34:51.000112 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:51.000118 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:51.000339 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:51.000527 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:34:51.000830 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:51.002323 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:51.003575 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:51.004410 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:51.005162 (+ 752us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:51.005311 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:51.005359 (+ 48us) server_negotiation.cc:299] Negotiation successful
0926 16:34:51.005440 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1768,"thread_start_us":193,"threads_started":1}
I20260926 16:34:51.005777 25704 tablet_bootstrap.cc:492] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce: No bootstrap required, opened a new log
I20260926 16:34:51.005853 25704 ts_tablet_manager.cc:1397] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260926 16:34:51.006260 25704 raft_consensus.cc:348] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3760e55349542ddaf855c0fe60d782c" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33957 } } peers { permanent_uuid: "9f3248e0f8d444edb82c65ff34e884a9" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34133 } } peers { permanent_uuid: "7a6306c756d14d899623fc8b45e187ce" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46507 } }
I20260926 16:34:51.006393 25704 raft_consensus.cc:374] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:34:51.006428 25704 raft_consensus.cc:729] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a6306c756d14d899623fc8b45e187ce, State: Initialized, Role: FOLLOWER
I20260926 16:34:51.006798 25704 consensus_queue.cc:260] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3760e55349542ddaf855c0fe60d782c" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33957 } } peers { permanent_uuid: "9f3248e0f8d444edb82c65ff34e884a9" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34133 } } peers { permanent_uuid: "7a6306c756d14d899623fc8b45e187ce" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46507 } }
I20260926 16:34:51.007203 25704 ts_tablet_manager.cc:1428] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260926 16:34:51.008806 26093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:50.998237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:34133 (local address 127.0.0.1:32878)
0926 16:34:51.003385 (+ 5148us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:51.003399 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:34:51.003487 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:34:51.004201 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:34:51.004204 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:34:51.004224 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:34:51.004309 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:51.004315 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:51.005673 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:51.005677 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:51.006832 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:51.006843 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:51.008424 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:51.008430 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:51.008515 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:51.008538 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 16:34:51.008608 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 16:34:51.008685 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5080,"thread_start_us":100,"threads_started":1}
I20260926 16:34:51.008406 26017 tablet_service.cc:1467] Processing CreateTablet for tablet 72d0a9d902804b2c9f418b45c3196008 (DEFAULT_TABLE table=test-table [id=fd2bf7b293834f9bbbc64ee5580cd74d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:34:51.009232 26094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:50.998577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32878 (local address 127.11.8.194:34133)
0926 16:34:50.998993 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:50.999005 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:51.003601 (+ 4596us) server_negotiation.cc:407] Connection header received
0926 16:34:51.003686 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:51.003692 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:51.003899 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:51.004042 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:34:51.004441 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:51.005525 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:51.006963 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:51.008134 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:51.008235 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:51.008357 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:51.008973 (+ 616us) server_negotiation.cc:299] Negotiation successful
0926 16:34:51.009065 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":189,"threads_started":1}
I20260926 16:34:51.009516 26017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72d0a9d902804b2c9f418b45c3196008. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:34:51.011653 25885 tablet_service.cc:1467] Processing CreateTablet for tablet 72d0a9d902804b2c9f418b45c3196008 (DEFAULT_TABLE table=test-table [id=fd2bf7b293834f9bbbc64ee5580cd74d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:34:51.012538 25885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72d0a9d902804b2c9f418b45c3196008. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:34:51.016224 26096 tablet_bootstrap.cc:492] T 72d0a9d902804b2c9f418b45c3196008 P b3760e55349542ddaf855c0fe60d782c: Bootstrap starting.
I20260926 16:34:51.016578 25843 tablet_bootstrap.cc:492] T 72d0a9d902804b2c9f418b45c3196008 P 9f3248e0f8d444edb82c65ff34e884a9: Bootstrap starting.
I20260926 16:34:51.018524 25843 tablet_bootstrap.cc:654] T 72d0a9d902804b2c9f418b45c3196008 P 9f3248e0f8d444edb82c65ff34e884a9: Neither blocks nor log segments found. Creating new log.
I20260926 16:34:51.018868 26096 tablet_bootstrap.cc:654] T 72d0a9d902804b2c9f418b45c3196008 P b3760e55349542ddaf855c0fe60d782c: Neither blocks nor log segments found. Creating new log.
I20260926 16:34:51.019393 25843 log.cc:824] T 72d0a9d902804b2c9f418b45c3196008 P 9f3248e0f8d444edb82c65ff34e884a9: Log is configured to *not* fsync() on all Append() calls
I20260926 16:34:51.019670 26096 log.cc:824] T 72d0a9d902804b2c9f418b45c3196008 P b3760e55349542ddaf855c0fe60d782c: Log is configured to *not* fsync() on all Append() calls
I20260926 16:34:51.021193 25843 tablet_bootstrap.cc:492] T 72d0a9d902804b2c9f418b45c3196008 P 9f3248e0f8d444edb82c65ff34e884a9: No bootstrap required, opened a new log
I20260926 16:34:51.021373 25843 ts_tablet_manager.cc:1397] T 72d0a9d902804b2c9f418b45c3196008 P 9f3248e0f8d444edb82c65ff34e884a9: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260926 16:34:51.021399 26096 tablet_bootstrap.cc:492] T 72d0a9d902804b2c9f418b45c3196008 P b3760e55349542ddaf855c0fe60d782c: No bootstrap required, opened a new log
I20260926 16:34:51.021571 26096 ts_tablet_manager.cc:1397] T 72d0a9d902804b2c9f418b45c3196008 P b3760e55349542ddaf855c0fe60d782c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 16:34:51.024251 25843 raft_consensus.cc:348] T 72d0a9d902804b2c9f418b45c3196008 P 9f3248e0f8d444edb82c65ff34e884a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3760e55349542ddaf855c0fe60d782c" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33957 } } peers { permanent_uuid: "9f3248e0f8d444edb82c65ff34e884a9" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34133 } } peers { permanent_uuid: "7a6306c756d14d899623fc8b45e187ce" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46507 } }
I20260926 16:34:51.024277 26096 raft_consensus.cc:348] T 72d0a9d902804b2c9f418b45c3196008 P b3760e55349542ddaf855c0fe60d782c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3760e55349542ddaf855c0fe60d782c" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33957 } } peers { permanent_uuid: "9f3248e0f8d444edb82c65ff34e884a9" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34133 } } peers { permanent_uuid: "7a6306c756d14d899623fc8b45e187ce" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46507 } }
I20260926 16:34:51.024469 25843 raft_consensus.cc:374] T 72d0a9d902804b2c9f418b45c3196008 P 9f3248e0f8d444edb82c65ff34e884a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:34:51.024463 26096 raft_consensus.cc:374] T 72d0a9d902804b2c9f418b45c3196008 P b3760e55349542ddaf855c0fe60d782c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:34:51.024534 25843 raft_consensus.cc:729] T 72d0a9d902804b2c9f418b45c3196008 P 9f3248e0f8d444edb82c65ff34e884a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f3248e0f8d444edb82c65ff34e884a9, State: Initialized, Role: FOLLOWER
I20260926 16:34:51.024530 26096 raft_consensus.cc:729] T 72d0a9d902804b2c9f418b45c3196008 P b3760e55349542ddaf855c0fe60d782c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3760e55349542ddaf855c0fe60d782c, State: Initialized, Role: FOLLOWER
I20260926 16:34:51.024952 26096 consensus_queue.cc:260] T 72d0a9d902804b2c9f418b45c3196008 P b3760e55349542ddaf855c0fe60d782c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3760e55349542ddaf855c0fe60d782c" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33957 } } peers { permanent_uuid: "9f3248e0f8d444edb82c65ff34e884a9" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34133 } } peers { permanent_uuid: "7a6306c756d14d899623fc8b45e187ce" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46507 } }
I20260926 16:34:51.024996 25843 consensus_queue.cc:260] T 72d0a9d902804b2c9f418b45c3196008 P 9f3248e0f8d444edb82c65ff34e884a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3760e55349542ddaf855c0fe60d782c" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33957 } } peers { permanent_uuid: "9f3248e0f8d444edb82c65ff34e884a9" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34133 } } peers { permanent_uuid: "7a6306c756d14d899623fc8b45e187ce" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46507 } }
I20260926 16:34:51.025682 26083 heartbeater.cc:503] Master 127.11.8.254:46389 was elected leader, sending a full tablet report...
I20260926 16:34:51.025943 26096 ts_tablet_manager.cc:1428] T 72d0a9d902804b2c9f418b45c3196008 P b3760e55349542ddaf855c0fe60d782c: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260926 16:34:51.026222 25843 ts_tablet_manager.cc:1428] T 72d0a9d902804b2c9f418b45c3196008 P 9f3248e0f8d444edb82c65ff34e884a9: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260926 16:34:51.026368 25951 heartbeater.cc:503] Master 127.11.8.254:46389 was elected leader, sending a full tablet report...
I20260926 16:34:51.050457 26100 raft_consensus.cc:482] T 72d0a9d902804b2c9f418b45c3196008 P 9f3248e0f8d444edb82c65ff34e884a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:34:51.050684 26100 raft_consensus.cc:504] T 72d0a9d902804b2c9f418b45c3196008 P 9f3248e0f8d444edb82c65ff34e884a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3760e55349542ddaf855c0fe60d782c" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33957 } } peers { permanent_uuid: "9f3248e0f8d444edb82c65ff34e884a9" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34133 } } peers { permanent_uuid: "7a6306c756d14d899623fc8b45e187ce" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46507 } }
I20260926 16:34:51.051889 26100 leader_election.cc:288] T 72d0a9d902804b2c9f418b45c3196008 P 9f3248e0f8d444edb82c65ff34e884a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3760e55349542ddaf855c0fe60d782c (127.11.8.195:33957), 7a6306c756d14d899623fc8b45e187ce (127.11.8.193:46507)
I20260926 16:34:51.055873 25953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:51.052148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:46507 (local address 127.11.8.194:60751)
0926 16:34:51.052375 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:51.052390 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:34:51.052482 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:34:51.052827 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:34:51.052830 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:34:51.052847 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:34:51.052980 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:51.052987 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:51.053797 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:51.053802 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:51.054762 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:51.054771 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:51.055454 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:51.055457 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:51.055564 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:51.055608 (+ 44us) client_negotiation.cc:769] Sending connection context
0926 16:34:51.055664 (+ 56us) client_negotiation.cc:240] Negotiation successful
0926 16:34:51.055740 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":111}
I20260926 16:34:51.055892 25820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:51.052235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:60751 (local address 127.11.8.193:46507)
0926 16:34:51.052405 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:51.052412 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:51.052432 (+ 20us) server_negotiation.cc:407] Connection header received
0926 16:34:51.052613 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:51.052617 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:51.052694 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:51.052801 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:34:51.053097 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:51.053680 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:51.054893 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:51.055323 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:51.055443 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:51.055594 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:51.055682 (+ 88us) server_negotiation.cc:299] Negotiation successful
0926 16:34:51.055772 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260926 16:34:51.055938 26101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:51.052177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:33957 (local address 127.11.8.194:40843)
0926 16:34:51.052600 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:51.052614 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:34:51.052711 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:34:51.052992 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:34:51.052995 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:34:51.053008 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:34:51.053139 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:51.053145 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:51.053858 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:51.053860 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:51.054776 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:51.054785 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:51.055482 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:51.055486 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:51.055641 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:51.055671 (+ 30us) client_negotiation.cc:769] Sending connection context
0926 16:34:51.055744 (+ 73us) client_negotiation.cc:240] Negotiation successful
0926 16:34:51.055805 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"mutex_wait_us":66,"thread_start_us":141,"threads_started":1}
I20260926 16:34:51.055979 26092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:51.052260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:40843 (local address 127.11.8.195:33957)
0926 16:34:51.052379 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:51.052384 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:51.052708 (+ 324us) server_negotiation.cc:407] Connection header received
0926 16:34:51.052833 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:51.052836 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:51.052897 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:51.052975 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:34:51.053252 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:51.053718 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:51.054949 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:51.055341 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:51.055480 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:51.055535 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:51.055784 (+ 249us) server_negotiation.cc:299] Negotiation successful
0926 16:34:51.055860 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 16:34:51.056533 25764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72d0a9d902804b2c9f418b45c3196008" candidate_uuid: "9f3248e0f8d444edb82c65ff34e884a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a6306c756d14d899623fc8b45e187ce" is_pre_election: true
I20260926 16:34:51.056852 25764 raft_consensus.cc:2454] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f3248e0f8d444edb82c65ff34e884a9 in term 0.
I20260926 16:34:51.056799 26037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72d0a9d902804b2c9f418b45c3196008" candidate_uuid: "9f3248e0f8d444edb82c65ff34e884a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3760e55349542ddaf855c0fe60d782c" is_pre_election: true
I20260926 16:34:51.057075 26037 raft_consensus.cc:2454] T 72d0a9d902804b2c9f418b45c3196008 P b3760e55349542ddaf855c0fe60d782c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f3248e0f8d444edb82c65ff34e884a9 in term 0.
I20260926 16:34:51.057287 25839 leader_election.cc:302] T 72d0a9d902804b2c9f418b45c3196008 P 9f3248e0f8d444edb82c65ff34e884a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a6306c756d14d899623fc8b45e187ce, 9f3248e0f8d444edb82c65ff34e884a9; no voters:
I20260926 16:34:51.057564 26100 raft_consensus.cc:2781] T 72d0a9d902804b2c9f418b45c3196008 P 9f3248e0f8d444edb82c65ff34e884a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:34:51.057646 26100 raft_consensus.cc:482] T 72d0a9d902804b2c9f418b45c3196008 P 9f3248e0f8d444edb82c65ff34e884a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:34:51.057689 26100 raft_consensus.cc:3037] T 72d0a9d902804b2c9f418b45c3196008 P 9f3248e0f8d444edb82c65ff34e884a9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:34:51.058704 26100 raft_consensus.cc:504] T 72d0a9d902804b2c9f418b45c3196008 P 9f3248e0f8d444edb82c65ff34e884a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3760e55349542ddaf855c0fe60d782c" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33957 } } peers { permanent_uuid: "9f3248e0f8d444edb82c65ff34e884a9" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34133 } } peers { permanent_uuid: "7a6306c756d14d899623fc8b45e187ce" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46507 } }
I20260926 16:34:51.059005 26100 leader_election.cc:288] T 72d0a9d902804b2c9f418b45c3196008 P 9f3248e0f8d444edb82c65ff34e884a9 [CANDIDATE]: Term 1 election: Requested vote from peers b3760e55349542ddaf855c0fe60d782c (127.11.8.195:33957), 7a6306c756d14d899623fc8b45e187ce (127.11.8.193:46507)
I20260926 16:34:51.059407 25764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72d0a9d902804b2c9f418b45c3196008" candidate_uuid: "9f3248e0f8d444edb82c65ff34e884a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a6306c756d14d899623fc8b45e187ce"
I20260926 16:34:51.059403 26037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72d0a9d902804b2c9f418b45c3196008" candidate_uuid: "9f3248e0f8d444edb82c65ff34e884a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3760e55349542ddaf855c0fe60d782c"
I20260926 16:34:51.059549 25764 raft_consensus.cc:3037] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:34:51.059551 26037 raft_consensus.cc:3037] T 72d0a9d902804b2c9f418b45c3196008 P b3760e55349542ddaf855c0fe60d782c [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:34:51.060802 25764 raft_consensus.cc:2454] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f3248e0f8d444edb82c65ff34e884a9 in term 1.
I20260926 16:34:51.060802 26037 raft_consensus.cc:2454] T 72d0a9d902804b2c9f418b45c3196008 P b3760e55349542ddaf855c0fe60d782c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f3248e0f8d444edb82c65ff34e884a9 in term 1.
I20260926 16:34:51.061127 25839 leader_election.cc:302] T 72d0a9d902804b2c9f418b45c3196008 P 9f3248e0f8d444edb82c65ff34e884a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a6306c756d14d899623fc8b45e187ce, 9f3248e0f8d444edb82c65ff34e884a9; no voters:
I20260926 16:34:51.061340 26100 raft_consensus.cc:2781] T 72d0a9d902804b2c9f418b45c3196008 P 9f3248e0f8d444edb82c65ff34e884a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:34:51.061553 26100 raft_consensus.cc:686] T 72d0a9d902804b2c9f418b45c3196008 P 9f3248e0f8d444edb82c65ff34e884a9 [term 1 LEADER]: Becoming Leader. State: Replica: 9f3248e0f8d444edb82c65ff34e884a9, State: Running, Role: LEADER
I20260926 16:34:51.061903 26100 consensus_queue.cc:237] T 72d0a9d902804b2c9f418b45c3196008 P 9f3248e0f8d444edb82c65ff34e884a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3760e55349542ddaf855c0fe60d782c" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33957 } } peers { permanent_uuid: "9f3248e0f8d444edb82c65ff34e884a9" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34133 } } peers { permanent_uuid: "7a6306c756d14d899623fc8b45e187ce" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46507 } }
I20260926 16:34:51.065125 25623 catalog_manager.cc:5495] T 72d0a9d902804b2c9f418b45c3196008 P 9f3248e0f8d444edb82c65ff34e884a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f3248e0f8d444edb82c65ff34e884a9 (127.11.8.194). New cstate: current_term: 1 leader_uuid: "9f3248e0f8d444edb82c65ff34e884a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3760e55349542ddaf855c0fe60d782c" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f3248e0f8d444edb82c65ff34e884a9" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34133 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a6306c756d14d899623fc8b45e187ce" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46507 } health_report { overall_health: UNKNOWN } } }
W20260926 16:34:51.069674 25813 tablet.cc:2369] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:34:51.079826 26094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:51.076723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32892 (local address 127.11.8.194:34133)
0926 16:34:51.076848 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:51.076851 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:51.076877 (+ 26us) server_negotiation.cc:407] Connection header received
0926 16:34:51.076987 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:51.076990 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:51.077037 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:51.077112 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:34:51.077286 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:51.077743 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:51.078558 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:51.078770 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:51.078844 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:51.079078 (+ 234us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:34:51.079449 (+ 371us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:34:51.079532 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:51.079586 (+ 54us) server_negotiation.cc:299] Negotiation successful
0926 16:34:51.079629 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 16:34:51.085018 25764 raft_consensus.cc:1261] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [term 1 FOLLOWER]: Refusing update from remote peer 9f3248e0f8d444edb82c65ff34e884a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:34:51.085039 26037 raft_consensus.cc:1261] T 72d0a9d902804b2c9f418b45c3196008 P b3760e55349542ddaf855c0fe60d782c [term 1 FOLLOWER]: Refusing update from remote peer 9f3248e0f8d444edb82c65ff34e884a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:34:51.085667 26102 consensus_queue.cc:1035] T 72d0a9d902804b2c9f418b45c3196008 P 9f3248e0f8d444edb82c65ff34e884a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a6306c756d14d899623fc8b45e187ce" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:34:51.085841 26100 consensus_queue.cc:1035] T 72d0a9d902804b2c9f418b45c3196008 P 9f3248e0f8d444edb82c65ff34e884a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3760e55349542ddaf855c0fe60d782c" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:34:51.090626 26104 mvcc.cc:205] Tried to move back new op lower bound from 7333644251478298624 to 7333644251392991232. Current Snapshot: MvccSnapshot[applied={T|T < 7333644251478298624}]
I20260926 16:34:51.091538 26105 mvcc.cc:205] Tried to move back new op lower bound from 7333644251478298624 to 7333644251392991232. Current Snapshot: MvccSnapshot[applied={T|T < 7333644251478298624}]
I20260926 16:34:51.095661 26106 mvcc.cc:205] Tried to move back new op lower bound from 7333644251478298624 to 7333644251392991232. Current Snapshot: MvccSnapshot[applied={T|T < 7333644251478298624}]
I20260926 16:34:51.099531 25624 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:47858:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:34:51.099697 25624 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:47858:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:34:51.101945 25624 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158: Sending DeleteTablet for 3 replicas of tablet 72d0a9d902804b2c9f418b45c3196008
I20260926 16:34:51.102628 25885 tablet_service.cc:1514] Processing DeleteTablet for tablet 72d0a9d902804b2c9f418b45c3196008 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:34:51 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:32878
I20260926 16:34:51.102641 26017 tablet_service.cc:1514] Processing DeleteTablet for tablet 72d0a9d902804b2c9f418b45c3196008 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:34:51 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:34384
I20260926 16:34:51.102869 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 25662
I20260926 16:34:51.103596 26113 tablet_replica.cc:330] T 72d0a9d902804b2c9f418b45c3196008 P b3760e55349542ddaf855c0fe60d782c: stopping tablet replica
I20260926 16:34:51.103860 26113 raft_consensus.cc:2229] T 72d0a9d902804b2c9f418b45c3196008 P b3760e55349542ddaf855c0fe60d782c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:34:51.104118 26113 raft_consensus.cc:2258] T 72d0a9d902804b2c9f418b45c3196008 P b3760e55349542ddaf855c0fe60d782c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:34:51.105599 26113 ts_tablet_manager.cc:1905] T 72d0a9d902804b2c9f418b45c3196008 P b3760e55349542ddaf855c0fe60d782c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:34:51.106976 26114 tablet_replica.cc:330] T 72d0a9d902804b2c9f418b45c3196008 P 9f3248e0f8d444edb82c65ff34e884a9: stopping tablet replica
I20260926 16:34:51.107298 26114 raft_consensus.cc:2229] T 72d0a9d902804b2c9f418b45c3196008 P 9f3248e0f8d444edb82c65ff34e884a9 [term 1 LEADER]: Raft consensus shutting down.
I20260926 16:34:51.107663 26114 raft_consensus.cc:2258] T 72d0a9d902804b2c9f418b45c3196008 P 9f3248e0f8d444edb82c65ff34e884a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:34:51.108420 26113 ts_tablet_manager.cc:1918] T 72d0a9d902804b2c9f418b45c3196008 P b3760e55349542ddaf855c0fe60d782c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:34:51.108551 26113 log.cc:1196] T 72d0a9d902804b2c9f418b45c3196008 P b3760e55349542ddaf855c0fe60d782c: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/wal/wals/72d0a9d902804b2c9f418b45c3196008
I20260926 16:34:51.108906 26113 ts_tablet_manager.cc:1939] T 72d0a9d902804b2c9f418b45c3196008 P b3760e55349542ddaf855c0fe60d782c: Deleting consensus metadata
I20260926 16:34:51.109002 26114 ts_tablet_manager.cc:1905] T 72d0a9d902804b2c9f418b45c3196008 P 9f3248e0f8d444edb82c65ff34e884a9: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 16:34:51.109562 25610 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.8.193:46507 (error 108)
I20260926 16:34:51.110234 25608 catalog_manager.cc:4841] TS b3760e55349542ddaf855c0fe60d782c (127.11.8.195:33957): tablet 72d0a9d902804b2c9f418b45c3196008 (table test-table [id=fd2bf7b293834f9bbbc64ee5580cd74d]) successfully deleted
I20260926 16:34:51.110680 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 25816
I20260926 16:34:51.111991 26093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:51.111690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:46507 (local address 127.0.0.1:44838)
0926 16:34:51.111830 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:51.111896 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.193:46507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260926 16:34:51.112155 25610 catalog_manager.cc:4584] TS 7a6306c756d14d899623fc8b45e187ce (127.11.8.193:46507): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 72d0a9d902804b2c9f418b45c3196008: Network error: Client connection negotiation failed: client connection to 127.11.8.193:46507: connect: Connection refused (error 111)
I20260926 16:34:51.116739 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 25954
I20260926 16:34:51.117717 26093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:51.116982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:34133 (local address 127.0.0.1:32896)
0926 16:34:51.117568 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:51.117630 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.194:34133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":492}
I20260926 16:34:51.122062 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 25591
I20260926 16:34:51.128326 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:46389
--webserver_interface=127.11.8.254
--webserver_port=34519
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:39665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:46389
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:34:51.235594 26116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:34:51.235846 26116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:34:51.235884 26116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:34:51.239862 26116 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:34:51.239946 26116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:34:51.239969 26116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:34:51.240007 26116 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:34:51.240036 26116 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:34:51.245186 26116 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:39665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:46389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:46389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=34519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26116
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:34:51.246182 26116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:34:51.247192 26116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:34:51.255273 26122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:34:51.255249 26121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:34:51.255280 26124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:34:51.255318 26116 server_base.cc:1034] running on GCE node
I20260926 16:34:51.256037 26116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:34:51.257066 26116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:34:51.258224 26116 hybrid_clock.cc:648] HybridClock initialized: now 1790440491258208 us; error 26 us; skew 500 ppm
Sep 26 16:34:51 dist-test-slave-2zvq krb5kdc[25043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440491, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:34:51.260972 26116 init.cc:377] Logged in from keytab as kudu/127.11.8.254@KRBTEST.COM (short username kudu)
I20260926 16:34:51.261786 26116 webserver.cc:460] Webserver started at http://127.11.8.254:34519/ using document root <none> and password file <none>
I20260926 16:34:51.262377 26116 fs_manager.cc:359] Metadata directory not provided
I20260926 16:34:51.262465 26116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:34:51.266057 26116 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:34:51.268175 26131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:51.269407 26116 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:34:51.269538 26116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "620a6df881a040c7b5d5266f54fb2158"
format_stamp: "Formatted at 2026-09-26 16:34:49 on dist-test-slave-2zvq"
server_key: "7d6281c966055fc1ecb90d7a8cb0a8b1"
server_key_iv: "d8240f03fa1092e08882b6407ccad068"
server_key_version: "encryptionkey@0"
I20260926 16:34:51.270067 26116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:34:51.287173 26116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:34:51.289970 26116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:34:51.290159 26116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:34:51.297055 26116 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:46389
I20260926 16:34:51.297075 26183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:46389 every 8 connection(s)
I20260926 16:34:51.298069 26116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:34:51.302047 26184 sys_catalog.cc:263] Verifying existing consensus state
I20260926 16:34:51.303646 26184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158: Bootstrap starting.
I20260926 16:34:51.304237 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 26116
I20260926 16:34:51.304658 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.193:46507
--local_ip_for_outbound_sockets=127.11.8.193
--tserver_master_addrs=127.11.8.254:46389
--webserver_port=46449
--webserver_interface=127.11.8.193
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:39665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 16:34:51.320658 26184 log.cc:824] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158: Log is configured to *not* fsync() on all Append() calls
I20260926 16:34:51.331668 26184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 16:34:51.332023 26184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158: Bootstrap complete.
I20260926 16:34:51.336133 26184 raft_consensus.cc:348] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "620a6df881a040c7b5d5266f54fb2158" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 46389 } }
I20260926 16:34:51.336544 26184 raft_consensus.cc:729] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 620a6df881a040c7b5d5266f54fb2158, State: Initialized, Role: FOLLOWER
I20260926 16:34:51.336999 26184 consensus_queue.cc:260] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "620a6df881a040c7b5d5266f54fb2158" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 46389 } }
I20260926 16:34:51.337141 26184 raft_consensus.cc:388] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:34:51.337231 26184 raft_consensus.cc:482] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:34:51.337349 26184 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [term 2 FOLLOWER]: Advancing to term 3
I20260926 16:34:51.339316 26184 raft_consensus.cc:504] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "620a6df881a040c7b5d5266f54fb2158" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 46389 } }
I20260926 16:34:51.339629 26184 leader_election.cc:302] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 620a6df881a040c7b5d5266f54fb2158; no voters:
I20260926 16:34:51.339947 26184 leader_election.cc:288] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260926 16:34:51.340029 26188 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [term 3 FOLLOWER]: Leader election won for term 3
I20260926 16:34:51.340235 26188 raft_consensus.cc:686] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [term 3 LEADER]: Becoming Leader. State: Replica: 620a6df881a040c7b5d5266f54fb2158, State: Running, Role: LEADER
I20260926 16:34:51.340607 26188 consensus_queue.cc:237] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "620a6df881a040c7b5d5266f54fb2158" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 46389 } }
I20260926 16:34:51.341145 26184 sys_catalog.cc:564] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:34:51.341907 26189 sys_catalog.cc:455] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "620a6df881a040c7b5d5266f54fb2158" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "620a6df881a040c7b5d5266f54fb2158" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 46389 } } }
I20260926 16:34:51.342037 26189 sys_catalog.cc:458] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [sys.catalog]: This master's current role is: LEADER
I20260926 16:34:51.342667 26190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 620a6df881a040c7b5d5266f54fb2158. Latest consensus state: current_term: 3 leader_uuid: "620a6df881a040c7b5d5266f54fb2158" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "620a6df881a040c7b5d5266f54fb2158" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 46389 } } }
I20260926 16:34:51.342808 26190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 620a6df881a040c7b5d5266f54fb2158 [sys.catalog]: This master's current role is: LEADER
I20260926 16:34:51.343286 26197 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:34:51.347400 26197 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:34:51.347929 26197 catalog_manager.cc:1259] Loaded cluster ID: bb4e410281e241999082312ec85add52
I20260926 16:34:51.348003 26197 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260926 16:34:51.348838 26197 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f38277929ad google::LogMessage::Fail() at ??:0
@ 0x7f382779588a google::LogMessage::SendToLog() at ??:0
@ 0x7f38277924de google::LogMessage::Flush() at ??:0
@ 0x7f3827793a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f38320d1fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f38320d2d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f38320cf579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f3832113640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f38299e43d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f38299e4c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f38299e63f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f38299d5512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f382b0196db start_thread at ??:0
@ 0x7f382629061f clone at ??:0
W20260926 16:34:51.454183 26186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:34:51.454481 26186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:34:51.454524 26186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:34:51.458544 26186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:34:51.458592 26186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:34:51.458674 26186 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.193
I20260926 16:34:51.463769 26186 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:39665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.193:46507
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.193
--webserver_port=46449
--tserver_master_addrs=127.11.8.254:46389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.193
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:34:51.464752 26186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:34:51.465766 26186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:34:51.475391 26226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:34:51.475394 26228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:34:51.475749 26186 server_base.cc:1034] running on GCE node
W20260926 16:34:51.475462 26225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:34:51.476162 26186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:34:51.476783 26186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:34:51.477970 26186 hybrid_clock.cc:648] HybridClock initialized: now 1790440491477919 us; error 68 us; skew 500 ppm
Sep 26 16:34:51 dist-test-slave-2zvq krb5kdc[25043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440491, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:34:51.480973 26186 init.cc:377] Logged in from keytab as kudu/127.11.8.193@KRBTEST.COM (short username kudu)
I20260926 16:34:51.481869 26186 webserver.cc:460] Webserver started at http://127.11.8.193:46449/ using document root <none> and password file <none>
I20260926 16:34:51.482498 26186 fs_manager.cc:359] Metadata directory not provided
I20260926 16:34:51.482589 26186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:34:51.486616 26186 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:34:51.488878 26235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:51.490175 26186 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:34:51.490332 26186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/wal
uuid: "7a6306c756d14d899623fc8b45e187ce"
format_stamp: "Formatted at 2026-09-26 16:34:49 on dist-test-slave-2zvq"
server_key: "0c4b5b14b95a43edce8654daee6cf0fa"
server_key_iv: "e8d2f45270d6fa44001e5e8f6d12009a"
server_key_version: "encryptionkey@0"
I20260926 16:34:51.490895 26186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:34:51.507862 26186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:34:51.511432 26186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:34:51.511639 26186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:34:51.512274 26186 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:34:51.513389 26242 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 16:34:51.518831 26186 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 16:34:51.518925 26186 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260926 16:34:51.519007 26186 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 16:34:51.520615 26186 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 16:34:51.520684 26186 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260926 16:34:51.520996 26242 tablet_bootstrap.cc:492] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce: Bootstrap starting.
I20260926 16:34:51.532311 26186 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.193:46507
I20260926 16:34:51.532338 26349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.193:46507 every 8 connection(s)
I20260926 16:34:51.533262 26186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
I20260926 16:34:51.541440 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 26186
I20260926 16:34:51.541913 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.194:34133
--local_ip_for_outbound_sockets=127.11.8.194
--tserver_master_addrs=127.11.8.254:46389
--webserver_port=45385
--webserver_interface=127.11.8.194
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:39665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 16:34:51.543318 26242 log.cc:824] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce: Log is configured to *not* fsync() on all Append() calls
I20260926 16:34:51.552235 26242 tablet_bootstrap.cc:492] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 16:34:51.552636 26242 tablet_bootstrap.cc:492] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce: Bootstrap complete.
I20260926 16:34:51.554052 26242 ts_tablet_manager.cc:1397] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce: Time spent bootstrapping tablet: real 0.033s user 0.025s sys 0.004s
I20260926 16:34:51.557657 26242 raft_consensus.cc:348] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3760e55349542ddaf855c0fe60d782c" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33957 } } peers { permanent_uuid: "9f3248e0f8d444edb82c65ff34e884a9" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34133 } } peers { permanent_uuid: "7a6306c756d14d899623fc8b45e187ce" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46507 } }
I20260926 16:34:51.558423 26242 raft_consensus.cc:729] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a6306c756d14d899623fc8b45e187ce, State: Initialized, Role: FOLLOWER
I20260926 16:34:51.558984 26242 consensus_queue.cc:260] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3760e55349542ddaf855c0fe60d782c" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33957 } } peers { permanent_uuid: "9f3248e0f8d444edb82c65ff34e884a9" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34133 } } peers { permanent_uuid: "7a6306c756d14d899623fc8b45e187ce" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46507 } }
I20260926 16:34:51.560216 26242 ts_tablet_manager.cc:1428] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce: Time spent starting tablet: real 0.006s user 0.009s sys 0.000s
W20260926 16:34:51.606736 26352 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:34:51.535491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.193:40723)
0926 16:34:51.535971 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:51.536022 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 16:34:51.536813 (+ 791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:34:51.605731 (+ 68918us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":141,"threads_started":1}
W20260926 16:34:51.607371 26350 heartbeater.cc:644] Failed to heartbeat to 127.11.8.254:46389 (0 consecutive failures): Network error: Failed to ping master at 127.11.8.254:46389: Client connection negotiation failed: client connection to 127.11.8.254:46389: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 16:34:51.608101 26352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:51.607831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.193:48207)
0926 16:34:51.607966 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:51.608038 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260926 16:34:51.608722 26352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:51.608521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.193:45167)
0926 16:34:51.608628 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:51.608677 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260926 16:34:51.608942 26350 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 16:34:51.659185 26353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:34:51.659503 26353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:34:51.659549 26353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:34:51.663631 26353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:34:51.663689 26353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:34:51.663780 26353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.194
I20260926 16:34:51.668987 26353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:39665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.194:34133
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.194
--webserver_port=45385
--tserver_master_addrs=127.11.8.254:46389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.194
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:34:51.670105 26353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:34:51.671120 26353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:34:51.680469 26361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:34:51.680475 26363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:34:51.680531 26360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:34:51.680665 26353 server_base.cc:1034] running on GCE node
I20260926 16:34:51.681145 26353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:34:51.681723 26353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:34:51.682899 26353 hybrid_clock.cc:648] HybridClock initialized: now 1790440491682878 us; error 34 us; skew 500 ppm
Sep 26 16:34:51 dist-test-slave-2zvq krb5kdc[25043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440491, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:34:51.685595 26353 init.cc:377] Logged in from keytab as kudu/127.11.8.194@KRBTEST.COM (short username kudu)
I20260926 16:34:51.686506 26353 webserver.cc:460] Webserver started at http://127.11.8.194:45385/ using document root <none> and password file <none>
I20260926 16:34:51.687040 26353 fs_manager.cc:359] Metadata directory not provided
I20260926 16:34:51.687129 26353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:34:51.690860 26353 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260926 16:34:51.692937 26370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:51.694123 26353 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:34:51.694348 26353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/wal
uuid: "9f3248e0f8d444edb82c65ff34e884a9"
format_stamp: "Formatted at 2026-09-26 16:34:49 on dist-test-slave-2zvq"
server_key: "cc4d9911c1a7a00ea1f1d467d1aa0c56"
server_key_iv: "ec7e67e6371902dcb3a82eeb64c2d647"
server_key_version: "encryptionkey@0"
I20260926 16:34:51.694939 26353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:34:51.710178 26353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:34:51.713022 26353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:34:51.713223 26353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:34:51.713785 26353 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:34:51.714864 26377 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260926 16:34:51.718688 26377 ts_tablet_manager.cc:1852] T 72d0a9d902804b2c9f418b45c3196008 P 9f3248e0f8d444edb82c65ff34e884a9: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260926 16:34:51.719192 26377 ts_tablet_manager.cc:1905] T 72d0a9d902804b2c9f418b45c3196008 P 9f3248e0f8d444edb82c65ff34e884a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:34:51.723081 26377 ts_tablet_manager.cc:1918] T 72d0a9d902804b2c9f418b45c3196008 P 9f3248e0f8d444edb82c65ff34e884a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:34:51.723177 26377 log.cc:1196] T 72d0a9d902804b2c9f418b45c3196008 P 9f3248e0f8d444edb82c65ff34e884a9: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/wal/wals/72d0a9d902804b2c9f418b45c3196008
I20260926 16:34:51.723536 26377 ts_tablet_manager.cc:1939] T 72d0a9d902804b2c9f418b45c3196008 P 9f3248e0f8d444edb82c65ff34e884a9: Deleting consensus metadata
I20260926 16:34:51.723902 26353 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260926 16:34:51.723975 26353 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.009s user 0.000s sys 0.000s
I20260926 16:34:51.724077 26353 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:34:51.724129 26353 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:51.733170 26353 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.194:34133
I20260926 16:34:51.733204 26484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.194:34133 every 8 connection(s)
I20260926 16:34:51.734057 26353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
I20260926 16:34:51.737574 26487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:51.736217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.194:55377)
0926 16:34:51.736762 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:51.736874 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":113,"threads_started":1}
W20260926 16:34:51.737995 26485 heartbeater.cc:644] Failed to heartbeat to 127.11.8.254:46389 (0 consecutive failures): Network error: Failed to ping master at 127.11.8.254:46389: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
I20260926 16:34:51.738641 26487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:51.738429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.194:37321)
0926 16:34:51.738535 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:51.738589 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260926 16:34:51.739233 26487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:51.739020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.194:37805)
0926 16:34:51.739134 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:51.739178 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260926 16:34:51.739403 26485 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 16:34:51.740674 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 26353
I20260926 16:34:51.741017 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.195:33957
--local_ip_for_outbound_sockets=127.11.8.195
--tserver_master_addrs=127.11.8.254:46389
--webserver_port=37475
--webserver_interface=127.11.8.195
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:39665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:34:51.851291 26488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:34:51.851553 26488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:34:51.851598 26488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:34:51.855533 26488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:34:51.855585 26488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:34:51.855643 26488 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.195
I20260926 16:34:51.860713 26488 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:39665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.195:33957
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.195
--webserver_port=37475
--tserver_master_addrs=127.11.8.254:46389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26488
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.195
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:34:51.861689 26488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:34:51.862739 26488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:34:51.872654 26493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:34:51.872678 26494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:34:51.872632 26496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:34:51.872828 26488 server_base.cc:1034] running on GCE node
I20260926 16:34:51.873395 26488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:34:51.873993 26488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:34:51.875174 26488 hybrid_clock.cc:648] HybridClock initialized: now 1790440491875135 us; error 50 us; skew 500 ppm
Sep 26 16:34:51 dist-test-slave-2zvq krb5kdc[25043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440491, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:34:51.878013 26488 init.cc:377] Logged in from keytab as kudu/127.11.8.195@KRBTEST.COM (short username kudu)
I20260926 16:34:51.878995 26488 webserver.cc:460] Webserver started at http://127.11.8.195:37475/ using document root <none> and password file <none>
I20260926 16:34:51.879585 26488 fs_manager.cc:359] Metadata directory not provided
I20260926 16:34:51.879680 26488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:34:51.883536 26488 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260926 16:34:51.885782 26503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:51.887110 26488 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260926 16:34:51.887274 26488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/wal
uuid: "b3760e55349542ddaf855c0fe60d782c"
format_stamp: "Formatted at 2026-09-26 16:34:49 on dist-test-slave-2zvq"
server_key: "9c82336090ac85ab12e89d56946fbad8"
server_key_iv: "2b166f5ad0f0fc921589dbc5bb521f90"
server_key_version: "encryptionkey@0"
I20260926 16:34:51.887818 26488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:34:51.901506 26488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:34:51.904641 26488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:34:51.904863 26488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:34:51.905471 26488 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:34:51.906378 26488 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:34:51.906442 26488 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:51.906510 26488 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:34:51.906558 26488 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:34:51.915772 26488 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.195:33957
I20260926 16:34:51.915800 26616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.195:33957 every 8 connection(s)
I20260926 16:34:51.916780 26488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
I20260926 16:34:51.917341 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 26488
I20260926 16:34:51.920548 26620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:51.918906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.195:59227)
0926 16:34:51.919434 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:51.919863 (+ 429us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":139,"threads_started":1}
W20260926 16:34:51.920984 26617 heartbeater.cc:644] Failed to heartbeat to 127.11.8.254:46389 (0 consecutive failures): Network error: Failed to ping master at 127.11.8.254:46389: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
I20260926 16:34:51.921526 26620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:51.921333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.195:47977)
0926 16:34:51.921429 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:51.921474 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260926 16:34:51.922081 26620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:51.921902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.195:59719)
0926 16:34:51.921993 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:51.922032 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260926 16:34:51.922364 26617 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 16:34:52.609977 26621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:52.609543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.193:48315)
0926 16:34:52.609807 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:52.609862 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":77,"threads_started":1}
I20260926 16:34:52.740288 26622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:52.739876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.194:50857)
0926 16:34:52.740155 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:52.740207 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":69,"threads_started":1}
I20260926 16:34:52.923321 26623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:52.922883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.195:54103)
0926 16:34:52.923134 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:52.923222 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":67,"threads_started":1}
I20260926 16:34:53.062825 26624 raft_consensus.cc:482] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:34:53.063089 26624 raft_consensus.cc:504] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3760e55349542ddaf855c0fe60d782c" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33957 } } peers { permanent_uuid: "9f3248e0f8d444edb82c65ff34e884a9" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34133 } } peers { permanent_uuid: "7a6306c756d14d899623fc8b45e187ce" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46507 } }
I20260926 16:34:53.064275 26624 leader_election.cc:288] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b3760e55349542ddaf855c0fe60d782c (127.11.8.195:33957), 9f3248e0f8d444edb82c65ff34e884a9 (127.11.8.194:34133)
Sep 26 16:34:53 dist-test-slave-2zvq krb5kdc[25043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440491, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for kudu/127.11.8.195@KRBTEST.COM
Sep 26 16:34:53 dist-test-slave-2zvq krb5kdc[25043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440491, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for kudu/127.11.8.194@KRBTEST.COM
I20260926 16:34:53.081271 26621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:53.064450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:33957 (local address 127.11.8.193:56735)
0926 16:34:53.064641 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:53.064655 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:34:53.064764 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:34:53.067093 (+ 2329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:34:53.067100 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:34:53.067706 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:34:53.068569 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:53.068589 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:53.070560 (+ 1971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:53.070564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:53.072200 (+ 1636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:53.072211 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:53.073210 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:53.073214 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:53.073600 (+ 386us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:53.074332 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:34:53.074350 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:34:53.076060 (+ 1710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:34:53.078553 (+ 2493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:53.078560 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:53.078574 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:53.078923 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:53.079478 (+ 555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:53.079482 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:53.079484 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:53.079631 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:53.080291 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:34:53.080297 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:34:53.080545 (+ 248us) client_negotiation.cc:769] Sending connection context
0926 16:34:53.080704 (+ 159us) client_negotiation.cc:240] Negotiation successful
0926 16:34:53.081062 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":497,"spinlock_wait_cycles":9216}
I20260926 16:34:53.081323 26626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:53.064634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:56735 (local address 127.11.8.195:33957)
0926 16:34:53.065042 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:53.065081 (+ 39us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:53.065140 (+ 59us) server_negotiation.cc:407] Connection header received
0926 16:34:53.066126 (+ 986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:53.066132 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:53.066613 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:53.066957 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:34:53.068775 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:53.070386 (+ 1611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:53.072341 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:53.073108 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:53.073246 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:53.076301 (+ 3055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:34:53.076306 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:34:53.076319 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:34:53.076345 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:34:53.078444 (+ 2099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:53.079138 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:53.079146 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:53.079154 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:53.079325 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:53.079784 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:53.079787 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:53.079789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:53.080161 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:34:53.080301 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:53.080846 (+ 545us) server_negotiation.cc:299] Negotiation successful
0926 16:34:53.081131 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":86,"threads_started":1}
I20260926 16:34:53.081375 26627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:53.064814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:37379 (local address 127.11.8.194:34133)
0926 16:34:53.065128 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0926 16:34:53.065138 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 16:34:53.065169 (+ 31us) server_negotiation.cc:407] Connection header received
0926 16:34:53.066105 (+ 936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:34:53.066116 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:34:53.066613 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:34:53.066957 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:34:53.068775 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:53.070386 (+ 1611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:53.072341 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:34:53.073062 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:34:53.073242 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:53.076582 (+ 3340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:34:53.076588 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:34:53.076599 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:34:53.076625 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:34:53.078452 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:53.079232 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:53.079237 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:53.079242 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:53.079373 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:34:53.079749 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:34:53.079753 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:34:53.079757 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:34:53.080146 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:34:53.080302 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0926 16:34:53.080922 (+ 620us) server_negotiation.cc:299] Negotiation successful
0926 16:34:53.081185 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":111,"threads_started":1}
I20260926 16:34:53.081394 26625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:53.064481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:34133 (local address 127.11.8.193:37379)
0926 16:34:53.064834 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:53.064848 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:34:53.064955 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:34:53.067093 (+ 2138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:34:53.067099 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:34:53.067706 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:34:53.068569 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:53.068589 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:53.070560 (+ 1971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:53.070564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:53.072201 (+ 1637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:34:53.072211 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:34:53.073210 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:34:53.073214 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:34:53.073481 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:34:53.074945 (+ 1464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:34:53.074961 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:34:53.076403 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:34:53.078580 (+ 2177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:53.078584 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:53.078586 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:53.078945 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:53.079532 (+ 587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:34:53.079535 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:34:53.079536 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:34:53.079633 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:34:53.080288 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:34:53.080294 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:34:53.080637 (+ 343us) client_negotiation.cc:769] Sending connection context
0926 16:34:53.080762 (+ 125us) client_negotiation.cc:240] Negotiation successful
0926 16:34:53.081196 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":591,"spinlock_wait_cycles":6144,"thread_start_us":74,"threads_started":1}
W20260926 16:34:53.084270 26237 leader_election.cc:341] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9f3248e0f8d444edb82c65ff34e884a9 (127.11.8.194:34133): Not found: Tablet not found: 72d0a9d902804b2c9f418b45c3196008
W20260926 16:34:53.084897 26236 leader_election.cc:341] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b3760e55349542ddaf855c0fe60d782c (127.11.8.195:33957): Not found: Tablet not found: 72d0a9d902804b2c9f418b45c3196008
I20260926 16:34:53.084990 26236 leader_election.cc:302] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a6306c756d14d899623fc8b45e187ce; no voters: 9f3248e0f8d444edb82c65ff34e884a9, b3760e55349542ddaf855c0fe60d782c
I20260926 16:34:53.085287 26624 raft_consensus.cc:2726] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 16:34:53.611271 26628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:53.610841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.193:58281)
0926 16:34:53.611112 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:53.611170 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":80,"threads_started":1}
I20260926 16:34:53.741499 26629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:53.741028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.194:52671)
0926 16:34:53.741329 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:53.741400 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":69,"threads_started":1}
I20260926 16:34:53.924405 26630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:53.924035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.195:48625)
0926 16:34:53.924264 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:53.924320 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":71,"threads_started":1}
I20260926 16:34:54.612396 26631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:54.611994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.193:45475)
0926 16:34:54.612229 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:54.612299 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":73,"threads_started":1}
I20260926 16:34:54.742584 26632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:54.742179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.194:49357)
0926 16:34:54.742445 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:54.742499 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":113,"threads_started":1}
I20260926 16:34:54.925458 26633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:54.925070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.195:37367)
0926 16:34:54.925305 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:54.925358 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":67,"threads_started":1}
I20260926 16:34:55.319718 26634 raft_consensus.cc:482] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:34:55.319868 26634 raft_consensus.cc:504] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3760e55349542ddaf855c0fe60d782c" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33957 } } peers { permanent_uuid: "9f3248e0f8d444edb82c65ff34e884a9" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34133 } } peers { permanent_uuid: "7a6306c756d14d899623fc8b45e187ce" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46507 } }
I20260926 16:34:55.320268 26634 leader_election.cc:288] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b3760e55349542ddaf855c0fe60d782c (127.11.8.195:33957), 9f3248e0f8d444edb82c65ff34e884a9 (127.11.8.194:34133)
W20260926 16:34:55.321039 26237 leader_election.cc:341] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9f3248e0f8d444edb82c65ff34e884a9 (127.11.8.194:34133): Not found: Tablet not found: 72d0a9d902804b2c9f418b45c3196008
W20260926 16:34:55.321128 26236 leader_election.cc:341] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b3760e55349542ddaf855c0fe60d782c (127.11.8.195:33957): Not found: Tablet not found: 72d0a9d902804b2c9f418b45c3196008
I20260926 16:34:55.321242 26236 leader_election.cc:302] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a6306c756d14d899623fc8b45e187ce; no voters: 9f3248e0f8d444edb82c65ff34e884a9, b3760e55349542ddaf855c0fe60d782c
I20260926 16:34:55.321404 26634 raft_consensus.cc:2726] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 16:34:55.613508 26635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:55.613106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.193:37521)
0926 16:34:55.613360 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:55.613414 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":71,"threads_started":1}
I20260926 16:34:55.743773 26636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:55.743313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.194:53681)
0926 16:34:55.743599 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:55.743668 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":88,"threads_started":1}
I20260926 16:34:55.926723 26637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:55.926344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.195:32809)
0926 16:34:55.926579 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:55.926630 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":81,"threads_started":1}
I20260926 16:34:56.614822 26638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:56.614421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.193:39099)
0926 16:34:56.614678 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:56.614729 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":72,"threads_started":1}
I20260926 16:34:56.744946 26639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:56.744536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.194:48715)
0926 16:34:56.744788 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:56.744855 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":72,"threads_started":1}
I20260926 16:34:56.927827 26640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:56.927437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.195:56627)
0926 16:34:56.927691 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:56.927743 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":72,"threads_started":1}
I20260926 16:34:57.615931 26641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:57.615548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.193:47955)
0926 16:34:57.615783 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:57.615836 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":73,"threads_started":1}
I20260926 16:34:57.746100 26642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:57.745667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.194:52915)
0926 16:34:57.745936 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:57.745994 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":77,"threads_started":1}
I20260926 16:34:57.929075 26643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:57.928666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.195:34603)
0926 16:34:57.928925 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:57.928982 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":94,"threads_started":1}
I20260926 16:34:58.054015 26644 raft_consensus.cc:482] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:34:58.054154 26644 raft_consensus.cc:504] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3760e55349542ddaf855c0fe60d782c" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33957 } } peers { permanent_uuid: "9f3248e0f8d444edb82c65ff34e884a9" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34133 } } peers { permanent_uuid: "7a6306c756d14d899623fc8b45e187ce" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46507 } }
I20260926 16:34:58.054673 26644 leader_election.cc:288] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b3760e55349542ddaf855c0fe60d782c (127.11.8.195:33957), 9f3248e0f8d444edb82c65ff34e884a9 (127.11.8.194:34133)
W20260926 16:34:58.055583 26237 leader_election.cc:341] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9f3248e0f8d444edb82c65ff34e884a9 (127.11.8.194:34133): Not found: Tablet not found: 72d0a9d902804b2c9f418b45c3196008
W20260926 16:34:58.055761 26236 leader_election.cc:341] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b3760e55349542ddaf855c0fe60d782c (127.11.8.195:33957): Not found: Tablet not found: 72d0a9d902804b2c9f418b45c3196008
I20260926 16:34:58.055867 26236 leader_election.cc:302] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a6306c756d14d899623fc8b45e187ce; no voters: 9f3248e0f8d444edb82c65ff34e884a9, b3760e55349542ddaf855c0fe60d782c
I20260926 16:34:58.056102 26644 raft_consensus.cc:2726] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 16:34:58.617051 26645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:58.616656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.193:50855)
0926 16:34:58.616899 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:58.616954 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":77,"threads_started":1}
I20260926 16:34:58.747304 26646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:58.746856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.194:40489)
0926 16:34:58.747160 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:58.747216 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":69,"threads_started":1}
I20260926 16:34:58.930302 26647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:58.929841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.195:39283)
0926 16:34:58.930112 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:58.930164 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":72,"threads_started":1}
I20260926 16:34:59.618258 26648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:59.617804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.193:32815)
0926 16:34:59.618063 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:59.618151 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":76,"threads_started":1}
I20260926 16:34:59.748514 26649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:59.748106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.194:36141)
0926 16:34:59.748361 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:59.748423 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":73,"threads_started":1}
I20260926 16:34:59.931491 26650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:34:59.931080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.195:36497)
0926 16:34:59.931339 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0926 16:34:59.931397 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":74,"threads_started":1}
I20260926 16:35:00.333293 26651 raft_consensus.cc:482] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:35:00.333469 26651 raft_consensus.cc:504] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3760e55349542ddaf855c0fe60d782c" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33957 } } peers { permanent_uuid: "9f3248e0f8d444edb82c65ff34e884a9" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34133 } } peers { permanent_uuid: "7a6306c756d14d899623fc8b45e187ce" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46507 } }
I20260926 16:35:00.333900 26651 leader_election.cc:288] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b3760e55349542ddaf855c0fe60d782c (127.11.8.195:33957), 9f3248e0f8d444edb82c65ff34e884a9 (127.11.8.194:34133)
W20260926 16:35:00.334715 26237 leader_election.cc:341] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9f3248e0f8d444edb82c65ff34e884a9 (127.11.8.194:34133): Not found: Tablet not found: 72d0a9d902804b2c9f418b45c3196008
W20260926 16:35:00.334901 26236 leader_election.cc:341] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b3760e55349542ddaf855c0fe60d782c (127.11.8.195:33957): Not found: Tablet not found: 72d0a9d902804b2c9f418b45c3196008
I20260926 16:35:00.334990 26236 leader_election.cc:302] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a6306c756d14d899623fc8b45e187ce; no voters: 9f3248e0f8d444edb82c65ff34e884a9, b3760e55349542ddaf855c0fe60d782c
I20260926 16:35:00.335183 26651 raft_consensus.cc:2726] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 16:35:00.619493 26652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:00.619064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.193:48247)
0926 16:35:00.619327 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:00.619395 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":101,"threads_started":1}
I20260926 16:35:00.749738 26653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:00.749319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.194:41999)
0926 16:35:00.749571 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:00.749629 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":76,"threads_started":1}
I20260926 16:35:00.932672 26654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:00.932245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.195:45163)
0926 16:35:00.932522 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:00.932577 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":114,"threads_started":1}
I20260926 16:35:01.620690 26655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:01.620294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.193:50505)
0926 16:35:01.620543 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:01.620596 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":76,"threads_started":1}
I20260926 16:35:01.750933 26656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:01.750502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.194:44139)
0926 16:35:01.750782 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:01.750838 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":80,"threads_started":1}
I20260926 16:35:01.933801 26657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:01.933395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.195:53635)
0926 16:35:01.933642 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:01.933694 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":73,"threads_started":1}
I20260926 16:35:02.621896 26658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:02.621402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.193:43771)
0926 16:35:02.621709 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:02.621793 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":93,"threads_started":1}
I20260926 16:35:02.752053 26659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:02.751650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.194:42875)
0926 16:35:02.751884 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:02.751936 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":76,"threads_started":1}
I20260926 16:35:02.935088 26660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:02.934666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.195:38181)
0926 16:35:02.934936 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:02.934995 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":71,"threads_started":1}
I20260926 16:35:03.623212 26661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:03.622764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.193:52329)
0926 16:35:03.623029 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:03.623089 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":78,"threads_started":1}
I20260926 16:35:03.753338 26662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:03.752843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.194:33011)
0926 16:35:03.753180 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:03.753239 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":124,"threads_started":1}
I20260926 16:35:03.918947 26663 raft_consensus.cc:482] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:35:03.919116 26663 raft_consensus.cc:504] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3760e55349542ddaf855c0fe60d782c" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33957 } } peers { permanent_uuid: "9f3248e0f8d444edb82c65ff34e884a9" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34133 } } peers { permanent_uuid: "7a6306c756d14d899623fc8b45e187ce" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46507 } }
I20260926 16:35:03.919546 26663 leader_election.cc:288] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b3760e55349542ddaf855c0fe60d782c (127.11.8.195:33957), 9f3248e0f8d444edb82c65ff34e884a9 (127.11.8.194:34133)
W20260926 16:35:03.920356 26236 leader_election.cc:341] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b3760e55349542ddaf855c0fe60d782c (127.11.8.195:33957): Not found: Tablet not found: 72d0a9d902804b2c9f418b45c3196008
W20260926 16:35:03.920449 26237 leader_election.cc:341] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9f3248e0f8d444edb82c65ff34e884a9 (127.11.8.194:34133): Not found: Tablet not found: 72d0a9d902804b2c9f418b45c3196008
I20260926 16:35:03.920600 26237 leader_election.cc:302] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a6306c756d14d899623fc8b45e187ce; no voters: 9f3248e0f8d444edb82c65ff34e884a9, b3760e55349542ddaf855c0fe60d782c
I20260926 16:35:03.920763 26663 raft_consensus.cc:2726] T 72d0a9d902804b2c9f418b45c3196008 P 7a6306c756d14d899623fc8b45e187ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 16:35:03.936273 26664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:03.935872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.195:60357)
0926 16:35:03.936120 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:03.936176 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":88,"threads_started":1}
I20260926 16:35:04.624393 26665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:04.623994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.193:40363)
0926 16:35:04.624239 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:04.624292 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":88,"threads_started":1}
I20260926 16:35:04.754566 26666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:04.754125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.194:43185)
0926 16:35:04.754421 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:04.754475 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":74,"threads_started":1}
I20260926 16:35:04.937374 26667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:04.936994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.195:49971)
0926 16:35:04.937229 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:04.937284 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":73,"threads_started":1}
I20260926 16:35:05.625614 26668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:05.625189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.193:53503)
0926 16:35:05.625474 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:05.625523 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":115,"threads_started":1}
I20260926 16:35:05.755748 26669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:05.755333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.194:44199)
0926 16:35:05.755576 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:05.755628 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":80,"threads_started":1}
I20260926 16:35:05.938519 26670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:05.938089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.195:55819)
0926 16:35:05.938377 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:05.938428 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":75,"threads_started":1}
I20260926 16:35:06.626890 26671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:06.626498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.193:39763)
0926 16:35:06.626741 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:06.626796 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":67,"threads_started":1}
I20260926 16:35:06.757118 26672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:06.756742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46389 (local address 127.11.8.194:52303)
0926 16:35:06.756974 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:06.757026 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":69,"threads_started":1}
I20260926 16:35:06.925215 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 26186
I20260926 16:35:06.930964 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 26353
I20260926 16:35:06.936383 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 26488
2026-09-26T16:35:06Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19006 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:35:06 dist-test-slave-2zvq krb5kdc[26675](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:35:06 dist-test-slave-2zvq krb5kdc[26675](info): set up 2 sockets
Sep 26 16:35:06 dist-test-slave-2zvq krb5kdc[26675](info): commencing operation
krb5kdc: starting...
W20260926 16:35:07.988291 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:35:08 dist-test-slave-2zvq krb5kdc[26675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:35:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:35:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
I20260926 16:35:08.109364 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:37989
--webserver_interface=127.11.8.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:38229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:37989
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:35:08.215173 26688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:35:08.215402 26688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:35:08.215440 26688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:35:08.219365 26688 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 16:35:08.219408 26688 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 16:35:08.219424 26688 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 16:35:08.219436 26688 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:35:08.219482 26688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:35:08.219518 26688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:35:08.219545 26688 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:35:08.219583 26688 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:35:08.224560 26688 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:38229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:37989
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:37989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:35:08.225476 26688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:35:08.226435 26688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:35:08.234422 26693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:35:08.234431 26696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:35:08.234813 26688 server_base.cc:1034] running on GCE node
W20260926 16:35:08.235036 26694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:35:08.235744 26688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:35:08.236725 26688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:35:08.237903 26688 hybrid_clock.cc:648] HybridClock initialized: now 1790440508237877 us; error 35 us; skew 500 ppm
Sep 26 16:35:08 dist-test-slave-2zvq krb5kdc[26675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440508, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:35:08.240610 26688 init.cc:377] Logged in from keytab as kudu/127.11.8.254@KRBTEST.COM (short username kudu)
I20260926 16:35:08.241436 26688 webserver.cc:460] Webserver started at http://127.11.8.254:37907/ using document root <none> and password file <none>
I20260926 16:35:08.241930 26688 fs_manager.cc:359] Metadata directory not provided
I20260926 16:35:08.241986 26688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:35:08.242142 26688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:35:08.244249 26688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "d12c7ef7fd4a497bb2d352f4f42c610c"
format_stamp: "Formatted at 2026-09-26 16:35:08 on dist-test-slave-2zvq"
server_key: "bf68e0ad6d7e03cb20519faec7a0e272"
server_key_iv: "4640bd55d0858f7a4575c9e1292d46d9"
server_key_version: "encryptionkey@0"
I20260926 16:35:08.244792 26688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "d12c7ef7fd4a497bb2d352f4f42c610c"
format_stamp: "Formatted at 2026-09-26 16:35:08 on dist-test-slave-2zvq"
server_key: "bf68e0ad6d7e03cb20519faec7a0e272"
server_key_iv: "4640bd55d0858f7a4575c9e1292d46d9"
server_key_version: "encryptionkey@0"
I20260926 16:35:08.248382 26688 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260926 16:35:08.250626 26703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:35:08.251798 26688 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:35:08.251925 26688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "d12c7ef7fd4a497bb2d352f4f42c610c"
format_stamp: "Formatted at 2026-09-26 16:35:08 on dist-test-slave-2zvq"
server_key: "bf68e0ad6d7e03cb20519faec7a0e272"
server_key_iv: "4640bd55d0858f7a4575c9e1292d46d9"
server_key_version: "encryptionkey@0"
I20260926 16:35:08.252022 26688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:35:08.263736 26688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:35:08.266806 26688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:35:08.266996 26688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:35:08.273811 26688 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:37989
I20260926 16:35:08.274078 26755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:37989 every 8 connection(s)
I20260926 16:35:08.274761 26688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:35:08.277325 26756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:35:08.282480 26756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c: Bootstrap starting.
I20260926 16:35:08.284670 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 26688
I20260926 16:35:08.284813 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:35:08.284845 26756 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c: Neither blocks nor log segments found. Creating new log.
I20260926 16:35:08.285141 11299 external_mini_cluster.cc:1428] Setting key 9542ca87475429e10a7bb584ed8ac858
I20260926 16:35:08.285697 26756 log.cc:824] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c: Log is configured to *not* fsync() on all Append() calls
I20260926 16:35:08.287923 26756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c: No bootstrap required, opened a new log
I20260926 16:35:08.292152 26756 raft_consensus.cc:348] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d12c7ef7fd4a497bb2d352f4f42c610c" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37989 } }
I20260926 16:35:08.292390 26756 raft_consensus.cc:374] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:35:08.292429 26756 raft_consensus.cc:729] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d12c7ef7fd4a497bb2d352f4f42c610c, State: Initialized, Role: FOLLOWER
I20260926 16:35:08.292881 26756 consensus_queue.cc:260] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d12c7ef7fd4a497bb2d352f4f42c610c" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37989 } }
I20260926 16:35:08.293030 26756 raft_consensus.cc:388] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:35:08.293110 26756 raft_consensus.cc:482] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:35:08.293205 26756 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:35:08.294183 26756 raft_consensus.cc:504] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d12c7ef7fd4a497bb2d352f4f42c610c" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37989 } }
Sep 26 16:35:08 dist-test-slave-2zvq krb5kdc[26675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:35:08.294553 26756 leader_election.cc:302] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d12c7ef7fd4a497bb2d352f4f42c610c; no voters:
I20260926 16:35:08.294848 26756 leader_election.cc:288] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:35:08.294955 26761 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:35:08.295200 26761 raft_consensus.cc:686] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [term 1 LEADER]: Becoming Leader. State: Replica: d12c7ef7fd4a497bb2d352f4f42c610c, State: Running, Role: LEADER
I20260926 16:35:08.295617 26761 consensus_queue.cc:237] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d12c7ef7fd4a497bb2d352f4f42c610c" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37989 } }
I20260926 16:35:08.296077 26756 sys_catalog.cc:564] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:35:08.297479 26762 sys_catalog.cc:455] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d12c7ef7fd4a497bb2d352f4f42c610c. Latest consensus state: current_term: 1 leader_uuid: "d12c7ef7fd4a497bb2d352f4f42c610c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d12c7ef7fd4a497bb2d352f4f42c610c" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37989 } } }
I20260926 16:35:08.297613 26762 sys_catalog.cc:458] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [sys.catalog]: This master's current role is: LEADER
I20260926 16:35:08.297981 26770 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:35:08.298442 26763 sys_catalog.cc:455] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d12c7ef7fd4a497bb2d352f4f42c610c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d12c7ef7fd4a497bb2d352f4f42c610c" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37989 } } }
I20260926 16:35:08.298543 26763 sys_catalog.cc:458] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [sys.catalog]: This master's current role is: LEADER
I20260926 16:35:08.300854 26760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:08.286189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49744 (local address 127.11.8.254:37989)
0926 16:35:08.286665 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0926 16:35:08.286684 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0926 16:35:08.286723 (+ 39us) server_negotiation.cc:407] Connection header received
0926 16:35:08.287589 (+ 866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:35:08.287610 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:35:08.287986 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:35:08.288346 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:35:08.288995 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:08.290480 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:08.291299 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:08.291991 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:08.292137 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:08.295091 (+ 2954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:35:08.295098 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:35:08.295116 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:35:08.295154 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:35:08.297197 (+ 2043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:35:08.297694 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:35:08.297700 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:35:08.297706 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:35:08.297847 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:35:08.298135 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:35:08.298142 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:35:08.298146 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:35:08.298670 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:35:08.298872 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0926 16:35:08.299641 (+ 769us) server_negotiation.cc:299] Negotiation successful
0926 16:35:08.299948 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":125,"threads_started":1}
I20260926 16:35:08.301591 26770 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:35:08.306839 26770 catalog_manager.cc:1347] Generated new cluster ID: 4bccdb7bda82407f820ba4ef70913ba4
I20260926 16:35:08.306902 26770 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:35:08.334578 26770 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:35:08.335493 26770 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:35:08.343881 26770 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c: Generated new TSK 0
I20260926 16:35:08.344521 26770 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.8.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab.
Entry for principal kudu/127.11.8.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab.
I20260926 16:35:08.379480 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.193:0
--local_ip_for_outbound_sockets=127.11.8.193
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:37989
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:38229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:35:08.491590 26782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:35:08.491843 26782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:35:08.491886 26782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:35:08.495961 26782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:35:08.496014 26782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:35:08.496070 26782 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.193
I20260926 16:35:08.501329 26782 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:38229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:37989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.193
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:35:08.502358 26782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:35:08.503345 26782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:35:08.512825 26790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:35:08.513077 26782 server_base.cc:1034] running on GCE node
W20260926 16:35:08.512780 26787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:35:08.512780 26788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:35:08.513626 26782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:35:08.514240 26782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:35:08.515445 26782 hybrid_clock.cc:648] HybridClock initialized: now 1790440508515426 us; error 32 us; skew 500 ppm
Sep 26 16:35:08 dist-test-slave-2zvq krb5kdc[26675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440508, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:35:08.518165 26782 init.cc:377] Logged in from keytab as kudu/127.11.8.193@KRBTEST.COM (short username kudu)
I20260926 16:35:08.519143 26782 webserver.cc:460] Webserver started at http://127.11.8.193:38551/ using document root <none> and password file <none>
I20260926 16:35:08.519802 26782 fs_manager.cc:359] Metadata directory not provided
I20260926 16:35:08.519874 26782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:35:08.520121 26782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:35:08.522359 26782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/data/instance:
uuid: "082a50743594445aacfab4de6c7d914f"
format_stamp: "Formatted at 2026-09-26 16:35:08 on dist-test-slave-2zvq"
server_key: "04c87e70745a85f80eede664d0e6f031"
server_key_iv: "cba335d4e28f3cc8d729c4887234d341"
server_key_version: "encryptionkey@0"
I20260926 16:35:08.522920 26782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance:
uuid: "082a50743594445aacfab4de6c7d914f"
format_stamp: "Formatted at 2026-09-26 16:35:08 on dist-test-slave-2zvq"
server_key: "04c87e70745a85f80eede664d0e6f031"
server_key_iv: "cba335d4e28f3cc8d729c4887234d341"
server_key_version: "encryptionkey@0"
I20260926 16:35:08.526700 26782 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260926 16:35:08.529043 26797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:35:08.530372 26782 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:35:08.530499 26782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/wal
uuid: "082a50743594445aacfab4de6c7d914f"
format_stamp: "Formatted at 2026-09-26 16:35:08 on dist-test-slave-2zvq"
server_key: "04c87e70745a85f80eede664d0e6f031"
server_key_iv: "cba335d4e28f3cc8d729c4887234d341"
server_key_version: "encryptionkey@0"
I20260926 16:35:08.530606 26782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:35:08.553702 26782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:35:08.556886 26782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:35:08.557080 26782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:35:08.557649 26782 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:35:08.558528 26782 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:35:08.558576 26782 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:35:08.558617 26782 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:35:08.558638 26782 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:35:08.567619 26782 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.193:46177
I20260926 16:35:08.567643 26910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.193:46177 every 8 connection(s)
I20260926 16:35:08.568506 26782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
I20260926 16:35:08.576126 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 26782
I20260926 16:35:08.576263 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance
I20260926 16:35:08.576529 11299 external_mini_cluster.cc:1428] Setting key 2ee2545a5e70afd224c7cc4efaccda1b
Sep 26 16:35:08 dist-test-slave-2zvq krb5kdc[26675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440508, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:35:08.582165 26760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:08.570312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:39051 (local address 127.11.8.254:37989)
0926 16:35:08.570452 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0926 16:35:08.570457 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:35:08.571469 (+ 1012us) server_negotiation.cc:407] Connection header received
0926 16:35:08.572300 (+ 831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:35:08.572303 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:35:08.572349 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:35:08.572429 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:35:08.573629 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:08.574167 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:08.575319 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:08.575540 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:08.575622 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:08.578465 (+ 2843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:35:08.578472 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:35:08.578477 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:35:08.578511 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:35:08.580013 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:35:08.580611 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:35:08.580613 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:35:08.580614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:35:08.580684 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:35:08.581058 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:35:08.581060 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:35:08.581061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:35:08.581214 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:35:08.581282 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0926 16:35:08.581908 (+ 626us) server_negotiation.cc:299] Negotiation successful
0926 16:35:08.582006 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 16:35:08.583307 26913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:08.570705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.193:39051)
0926 16:35:08.571323 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:08.571364 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 16:35:08.572094 (+ 730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:35:08.572620 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:35:08.572627 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:35:08.573004 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:35:08.573473 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:35:08.573487 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:35:08.574302 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:35:08.574304 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:35:08.575179 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:35:08.575188 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:35:08.575666 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:35:08.575669 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:35:08.575816 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:08.576539 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:35:08.576562 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:35:08.578305 (+ 1743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:35:08.580125 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:35:08.580133 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:35:08.580149 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:35:08.580508 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:35:08.580778 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:35:08.580784 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:35:08.580789 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:35:08.580966 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:35:08.581311 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:35:08.581320 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:35:08.581633 (+ 313us) client_negotiation.cc:769] Sending connection context
0926 16:35:08.581843 (+ 210us) client_negotiation.cc:240] Negotiation successful
0926 16:35:08.582197 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":110,"threads_started":1}
I20260926 16:35:08.584553 26911 heartbeater.cc:347] Connected to a master server at 127.11.8.254:37989
I20260926 16:35:08.584810 26911 heartbeater.cc:464] Registering TS with master...
I20260926 16:35:08.585263 26911 heartbeater.cc:511] Master 127.11.8.254:37989 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.8.194@KRBTEST.COM; defaulting to no policy
I20260926 16:35:08.586745 26720 ts_manager.cc:194] Registered new tserver with Master: 082a50743594445aacfab4de6c7d914f (127.11.8.193:46177)
I20260926 16:35:08.587976 26720 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.193@KRBTEST.COM'} at 127.11.8.193:39051
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab.
Entry for principal kudu/127.11.8.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab.
I20260926 16:35:08.605244 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.194:0
--local_ip_for_outbound_sockets=127.11.8.194
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:37989
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:38229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:35:08.715188 26916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:35:08.715483 26916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:35:08.715559 26916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:35:08.719647 26916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:35:08.719723 26916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:35:08.719808 26916 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.194
I20260926 16:35:08.725080 26916 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:38229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:37989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.194
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:35:08.726092 26916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:35:08.727151 26916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:35:08.736662 26922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:35:08.736711 26921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:35:08.736711 26924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:35:08.736819 26916 server_base.cc:1034] running on GCE node
I20260926 16:35:08.737305 26916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:35:08.737895 26916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:35:08.739068 26916 hybrid_clock.cc:648] HybridClock initialized: now 1790440508739037 us; error 42 us; skew 500 ppm
Sep 26 16:35:08 dist-test-slave-2zvq krb5kdc[26675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440508, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:35:08.741716 26916 init.cc:377] Logged in from keytab as kudu/127.11.8.194@KRBTEST.COM (short username kudu)
I20260926 16:35:08.742599 26916 webserver.cc:460] Webserver started at http://127.11.8.194:38161/ using document root <none> and password file <none>
I20260926 16:35:08.743131 26916 fs_manager.cc:359] Metadata directory not provided
I20260926 16:35:08.743214 26916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:35:08.743453 26916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:35:08.745391 26916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/data/instance:
uuid: "c85733551b2d4dd7b984240ad8ab3224"
format_stamp: "Formatted at 2026-09-26 16:35:08 on dist-test-slave-2zvq"
server_key: "fb9e803b8efcad51e23c812fda8f78a8"
server_key_iv: "d3ebeccfddfb8d1249e0959af884e529"
server_key_version: "encryptionkey@0"
I20260926 16:35:08.745934 26916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance:
uuid: "c85733551b2d4dd7b984240ad8ab3224"
format_stamp: "Formatted at 2026-09-26 16:35:08 on dist-test-slave-2zvq"
server_key: "fb9e803b8efcad51e23c812fda8f78a8"
server_key_iv: "d3ebeccfddfb8d1249e0959af884e529"
server_key_version: "encryptionkey@0"
I20260926 16:35:08.749464 26916 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260926 16:35:08.752038 26931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:35:08.753190 26916 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:35:08.753337 26916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/wal
uuid: "c85733551b2d4dd7b984240ad8ab3224"
format_stamp: "Formatted at 2026-09-26 16:35:08 on dist-test-slave-2zvq"
server_key: "fb9e803b8efcad51e23c812fda8f78a8"
server_key_iv: "d3ebeccfddfb8d1249e0959af884e529"
server_key_version: "encryptionkey@0"
I20260926 16:35:08.753448 26916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:35:08.784960 26916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:35:08.787846 26916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:35:08.788050 26916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:35:08.788642 26916 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:35:08.789446 26916 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:35:08.789513 26916 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:35:08.789579 26916 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:35:08.789623 26916 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:35:08.798702 26916 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.194:34339
I20260926 16:35:08.798722 27044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.194:34339 every 8 connection(s)
I20260926 16:35:08.799598 26916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
I20260926 16:35:08.801753 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 26916
I20260926 16:35:08.801872 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance
I20260926 16:35:08.802244 11299 external_mini_cluster.cc:1428] Setting key d1b4aa11a4d6877bc816ab05f0a55282
Sep 26 16:35:08 dist-test-slave-2zvq krb5kdc[26675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440508, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.8.195@KRBTEST.COM; defaulting to no policy
I20260926 16:35:08.814383 26760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:08.801280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:54841 (local address 127.11.8.254:37989)
0926 16:35:08.801428 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0926 16:35:08.801432 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:35:08.802653 (+ 1221us) server_negotiation.cc:407] Connection header received
0926 16:35:08.803630 (+ 977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:35:08.803633 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:35:08.803687 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:35:08.803770 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:35:08.805347 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:08.805855 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:08.807678 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:08.807893 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:08.807998 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:08.810442 (+ 2444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:35:08.810449 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:35:08.810453 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:35:08.810485 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:35:08.812192 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:35:08.812698 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:35:08.812700 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:35:08.812702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:35:08.812812 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:35:08.813208 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:35:08.813212 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:35:08.813218 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:35:08.813402 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:35:08.813487 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 16:35:08.814034 (+ 547us) server_negotiation.cc:299] Negotiation successful
0926 16:35:08.814163 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.195@KRBTEST.COM" created.
I20260926 16:35:08.815263 27047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:08.801689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.194:54841)
0926 16:35:08.802495 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:08.802548 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 16:35:08.803451 (+ 903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:35:08.803951 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:35:08.803960 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:35:08.804506 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:35:08.805188 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:35:08.805210 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:35:08.805978 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:35:08.805984 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:35:08.807562 (+ 1578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:35:08.807577 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:35:08.808047 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:35:08.808051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:35:08.808219 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:08.808838 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:35:08.808854 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:35:08.810236 (+ 1382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:35:08.812309 (+ 2073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:35:08.812313 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:35:08.812324 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:35:08.812586 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:35:08.812901 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:35:08.812903 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:35:08.812904 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:35:08.813034 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:35:08.813517 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:35:08.813522 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:35:08.813760 (+ 238us) client_negotiation.cc:769] Sending connection context
0926 16:35:08.813918 (+ 158us) client_negotiation.cc:240] Negotiation successful
0926 16:35:08.814250 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":148,"threads_started":1}
I20260926 16:35:08.816473 27045 heartbeater.cc:347] Connected to a master server at 127.11.8.254:37989
I20260926 16:35:08.816740 27045 heartbeater.cc:464] Registering TS with master...
I20260926 16:35:08.817328 27045 heartbeater.cc:511] Master 127.11.8.254:37989 requested a full tablet report, sending...
I20260926 16:35:08.818440 26720 ts_manager.cc:194] Registered new tserver with Master: c85733551b2d4dd7b984240ad8ab3224 (127.11.8.194:34339)
I20260926 16:35:08.819015 26720 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.194@KRBTEST.COM'} at 127.11.8.194:54841
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab.
Entry for principal kudu/127.11.8.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab.
I20260926 16:35:08.830922 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.195:0
--local_ip_for_outbound_sockets=127.11.8.195
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:37989
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:38229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:35:08.940652 27050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:35:08.940934 27050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:35:08.941010 27050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:35:08.945009 27050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:35:08.945091 27050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:35:08.945176 27050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.195
I20260926 16:35:08.950354 27050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:38229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:37989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.195
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:35:08.951354 27050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:35:08.952348 27050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:35:08.961994 27058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:35:08.961990 27055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:35:08.962131 27056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:35:08.962487 27050 server_base.cc:1034] running on GCE node
I20260926 16:35:08.962914 27050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:35:08.963565 27050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:35:08.964766 27050 hybrid_clock.cc:648] HybridClock initialized: now 1790440508964725 us; error 52 us; skew 500 ppm
Sep 26 16:35:08 dist-test-slave-2zvq krb5kdc[26675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440508, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:35:08.967546 27050 init.cc:377] Logged in from keytab as kudu/127.11.8.195@KRBTEST.COM (short username kudu)
I20260926 16:35:08.968370 27050 webserver.cc:460] Webserver started at http://127.11.8.195:38133/ using document root <none> and password file <none>
I20260926 16:35:08.968878 27050 fs_manager.cc:359] Metadata directory not provided
I20260926 16:35:08.968935 27050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:35:08.969087 27050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:35:08.971069 27050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/data/instance:
uuid: "03003d7115c04b9d90c5c8d257aa5a33"
format_stamp: "Formatted at 2026-09-26 16:35:08 on dist-test-slave-2zvq"
server_key: "0a5993d3a7529ac1925c75221237d151"
server_key_iv: "3c2d75a4e0bb2641545b2e541fd7353b"
server_key_version: "encryptionkey@0"
I20260926 16:35:08.971576 27050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance:
uuid: "03003d7115c04b9d90c5c8d257aa5a33"
format_stamp: "Formatted at 2026-09-26 16:35:08 on dist-test-slave-2zvq"
server_key: "0a5993d3a7529ac1925c75221237d151"
server_key_iv: "3c2d75a4e0bb2641545b2e541fd7353b"
server_key_version: "encryptionkey@0"
I20260926 16:35:08.975100 27050 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 16:35:08.977505 27065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:35:08.978749 27050 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:35:08.978845 27050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/wal
uuid: "03003d7115c04b9d90c5c8d257aa5a33"
format_stamp: "Formatted at 2026-09-26 16:35:08 on dist-test-slave-2zvq"
server_key: "0a5993d3a7529ac1925c75221237d151"
server_key_iv: "3c2d75a4e0bb2641545b2e541fd7353b"
server_key_version: "encryptionkey@0"
I20260926 16:35:08.978926 27050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:35:09.007113 27050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:35:09.009891 27050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:35:09.010040 27050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:35:09.010597 27050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:35:09.011351 27050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:35:09.011397 27050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:35:09.011436 27050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:35:09.011458 27050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:35:09.020198 27050 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.195:38387
I20260926 16:35:09.020255 27178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.195:38387 every 8 connection(s)
I20260926 16:35:09.021083 27050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
I20260926 16:35:09.027366 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 27050
I20260926 16:35:09.027493 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance
I20260926 16:35:09.027765 11299 external_mini_cluster.cc:1428] Setting key 2073b9f98d78b0ebb8765f08381dfb7b
Sep 26 16:35:09 dist-test-slave-2zvq krb5kdc[26675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440508, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:35:09.034569 26760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:09.022940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:53125 (local address 127.11.8.254:37989)
0926 16:35:09.023118 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0926 16:35:09.023122 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:35:09.024122 (+ 1000us) server_negotiation.cc:407] Connection header received
0926 16:35:09.025017 (+ 895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:35:09.025020 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:35:09.025077 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:35:09.025174 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:35:09.026350 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:09.026870 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:09.028149 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:09.028466 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:09.028600 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:09.031092 (+ 2492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:35:09.031095 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:35:09.031097 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:35:09.031119 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:35:09.032587 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:35:09.033024 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:35:09.033027 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:35:09.033028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:35:09.033097 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:35:09.033404 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:35:09.033407 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:35:09.033408 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:35:09.033697 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:35:09.033793 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0926 16:35:09.034249 (+ 456us) server_negotiation.cc:299] Negotiation successful
0926 16:35:09.034406 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260926 16:35:09.035419 27181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:09.023290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.195:53125)
0926 16:35:09.023976 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:09.024016 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 16:35:09.024811 (+ 795us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:35:09.025346 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:35:09.025353 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:35:09.025689 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:35:09.026141 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:35:09.026154 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:35:09.026990 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:35:09.026994 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:35:09.027984 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:35:09.027994 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:35:09.028667 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:35:09.028674 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:35:09.028848 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:09.029393 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:35:09.029412 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:35:09.030938 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:35:09.032683 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:35:09.032688 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:35:09.032698 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:35:09.032939 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:35:09.033175 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:35:09.033178 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:35:09.033179 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:35:09.033311 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:35:09.033797 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:35:09.033802 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:35:09.034012 (+ 210us) client_negotiation.cc:769] Sending connection context
0926 16:35:09.034201 (+ 189us) client_negotiation.cc:240] Negotiation successful
0926 16:35:09.034547 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":116,"threads_started":1}
I20260926 16:35:09.036417 27179 heartbeater.cc:347] Connected to a master server at 127.11.8.254:37989
I20260926 16:35:09.036620 27179 heartbeater.cc:464] Registering TS with master...
I20260926 16:35:09.037030 27179 heartbeater.cc:511] Master 127.11.8.254:37989 requested a full tablet report, sending...
I20260926 16:35:09.038122 26720 ts_manager.cc:194] Registered new tserver with Master: 03003d7115c04b9d90c5c8d257aa5a33 (127.11.8.195:38387)
I20260926 16:35:09.038776 26720 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.195@KRBTEST.COM'} at 127.11.8.195:53125
I20260926 16:35:09.041731 11299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 16:35:09.051011 26760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:09.043750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49746 (local address 127.11.8.254:37989)
0926 16:35:09.043885 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0926 16:35:09.043889 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:35:09.044118 (+ 229us) server_negotiation.cc:407] Connection header received
0926 16:35:09.044174 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:35:09.044176 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:35:09.044213 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:35:09.044273 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:35:09.044906 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:09.045416 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:09.046362 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:09.046604 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:09.046696 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:09.048025 (+ 1329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:35:09.048028 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:35:09.048030 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:35:09.048051 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:35:09.049384 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:35:09.049774 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:35:09.049777 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:35:09.049778 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:35:09.049854 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:35:09.050102 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:35:09.050104 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:35:09.050106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:35:09.050408 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:35:09.050532 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0926 16:35:09.050712 (+ 180us) server_negotiation.cc:299] Negotiation successful
0926 16:35:09.050807 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 16:35:09.053771 26720 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49746:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:35:09.056869 26720 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:35:09.070632 27193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:09.065331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:46177 (local address 127.0.0.1:51104)
0926 16:35:09.066021 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:09.066033 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:35:09.066098 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:35:09.066699 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:35:09.066705 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:35:09.066729 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:35:09.066864 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:35:09.066870 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:35:09.067900 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:35:09.067906 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:35:09.068927 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:35:09.068934 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:35:09.070056 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:35:09.070059 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:35:09.070171 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:09.070206 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 16:35:09.070352 (+ 146us) client_negotiation.cc:240] Negotiation successful
0926 16:35:09.070472 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":86,"threads_started":1}
I20260926 16:35:09.070713 27194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:09.065818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51104 (local address 127.11.8.193:46177)
0926 16:35:09.066178 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0926 16:35:09.066195 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 16:35:09.066217 (+ 22us) server_negotiation.cc:407] Connection header received
0926 16:35:09.066325 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:35:09.066331 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:35:09.066540 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:35:09.066697 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:35:09.067001 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:09.067774 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:09.069089 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:09.069936 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:09.070056 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:09.070205 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0926 16:35:09.070479 (+ 274us) server_negotiation.cc:299] Negotiation successful
0926 16:35:09.070570 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":88,"threads_started":1}
I20260926 16:35:09.070887 27190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:09.065073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:34339 (local address 127.0.0.1:58452)
0926 16:35:09.065877 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:09.065942 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0926 16:35:09.066078 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:35:09.066863 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:35:09.066866 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:35:09.066881 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:35:09.066981 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:35:09.066987 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:35:09.067991 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:35:09.067996 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:35:09.069163 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:35:09.069173 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:35:09.070313 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:35:09.070319 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:35:09.070553 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:09.070576 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 16:35:09.070673 (+ 97us) client_negotiation.cc:240] Negotiation successful
0926 16:35:09.070747 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"thread_start_us":97,"threads_started":1}
I20260926 16:35:09.071378 27191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:09.065272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58452 (local address 127.11.8.194:34339)
0926 16:35:09.065661 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0926 16:35:09.065677 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 16:35:09.066211 (+ 534us) server_negotiation.cc:407] Connection header received
0926 16:35:09.066332 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:35:09.066337 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:35:09.066519 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:35:09.066676 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:35:09.067092 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:09.067860 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:09.069300 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:09.070140 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:09.070240 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:09.070394 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0926 16:35:09.071162 (+ 768us) server_negotiation.cc:299] Negotiation successful
0926 16:35:09.071233 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":96,"threads_started":1}
I20260926 16:35:09.072885 27192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:09.065784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:38387 (local address 127.0.0.1:37566)
0926 16:35:09.066955 (+ 1171us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:09.066968 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:35:09.067095 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:35:09.068363 (+ 1268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:35:09.068366 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:35:09.068384 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:35:09.068461 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:35:09.068467 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:35:09.069745 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:35:09.069748 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:35:09.071247 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:35:09.071257 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:35:09.072488 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:35:09.072492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:35:09.072584 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:09.072605 (+ 21us) client_negotiation.cc:769] Sending connection context
0926 16:35:09.072676 (+ 71us) client_negotiation.cc:240] Negotiation successful
0926 16:35:09.072752 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1108,"thread_start_us":92,"threads_started":1}
I20260926 16:35:09.073312 27195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:09.066331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37566 (local address 127.11.8.195:38387)
0926 16:35:09.067946 (+ 1615us) server_negotiation.cc:206] Beginning negotiation
0926 16:35:09.067961 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 16:35:09.067981 (+ 20us) server_negotiation.cc:407] Connection header received
0926 16:35:09.068047 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:35:09.068052 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:35:09.068213 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:35:09.068367 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:35:09.068555 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:09.069602 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:09.071372 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:09.072152 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:09.072280 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:09.072431 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0926 16:35:09.073117 (+ 686us) server_negotiation.cc:299] Negotiation successful
0926 16:35:09.073202 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":140,"threads_started":1}
I20260926 16:35:09.073592 26845 tablet_service.cc:1467] Processing CreateTablet for tablet 6bc7565d818741b5984d5c4f468d9864 (DEFAULT_TABLE table=test-table [id=2b9056629aa74348b4274a812331fda7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:35:09.074155 26979 tablet_service.cc:1467] Processing CreateTablet for tablet 6bc7565d818741b5984d5c4f468d9864 (DEFAULT_TABLE table=test-table [id=2b9056629aa74348b4274a812331fda7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:35:09.074780 26845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bc7565d818741b5984d5c4f468d9864. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:35:09.075184 26979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bc7565d818741b5984d5c4f468d9864. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:35:09.075583 27113 tablet_service.cc:1467] Processing CreateTablet for tablet 6bc7565d818741b5984d5c4f468d9864 (DEFAULT_TABLE table=test-table [id=2b9056629aa74348b4274a812331fda7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:35:09.076486 27113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bc7565d818741b5984d5c4f468d9864. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:35:09.082108 27197 tablet_bootstrap.cc:492] T 6bc7565d818741b5984d5c4f468d9864 P c85733551b2d4dd7b984240ad8ab3224: Bootstrap starting.
I20260926 16:35:09.083165 27196 tablet_bootstrap.cc:492] T 6bc7565d818741b5984d5c4f468d9864 P 03003d7115c04b9d90c5c8d257aa5a33: Bootstrap starting.
I20260926 16:35:09.084694 27197 tablet_bootstrap.cc:654] T 6bc7565d818741b5984d5c4f468d9864 P c85733551b2d4dd7b984240ad8ab3224: Neither blocks nor log segments found. Creating new log.
I20260926 16:35:09.085258 27196 tablet_bootstrap.cc:654] T 6bc7565d818741b5984d5c4f468d9864 P 03003d7115c04b9d90c5c8d257aa5a33: Neither blocks nor log segments found. Creating new log.
I20260926 16:35:09.085440 27197 log.cc:824] T 6bc7565d818741b5984d5c4f468d9864 P c85733551b2d4dd7b984240ad8ab3224: Log is configured to *not* fsync() on all Append() calls
I20260926 16:35:09.086009 27198 tablet_bootstrap.cc:492] T 6bc7565d818741b5984d5c4f468d9864 P 082a50743594445aacfab4de6c7d914f: Bootstrap starting.
I20260926 16:35:09.086035 27196 log.cc:824] T 6bc7565d818741b5984d5c4f468d9864 P 03003d7115c04b9d90c5c8d257aa5a33: Log is configured to *not* fsync() on all Append() calls
I20260926 16:35:09.087450 27197 tablet_bootstrap.cc:492] T 6bc7565d818741b5984d5c4f468d9864 P c85733551b2d4dd7b984240ad8ab3224: No bootstrap required, opened a new log
I20260926 16:35:09.087641 27197 ts_tablet_manager.cc:1397] T 6bc7565d818741b5984d5c4f468d9864 P c85733551b2d4dd7b984240ad8ab3224: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 16:35:09.087922 27196 tablet_bootstrap.cc:492] T 6bc7565d818741b5984d5c4f468d9864 P 03003d7115c04b9d90c5c8d257aa5a33: No bootstrap required, opened a new log
I20260926 16:35:09.088093 27196 ts_tablet_manager.cc:1397] T 6bc7565d818741b5984d5c4f468d9864 P 03003d7115c04b9d90c5c8d257aa5a33: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260926 16:35:09.088234 27198 tablet_bootstrap.cc:654] T 6bc7565d818741b5984d5c4f468d9864 P 082a50743594445aacfab4de6c7d914f: Neither blocks nor log segments found. Creating new log.
I20260926 16:35:09.088866 27198 log.cc:824] T 6bc7565d818741b5984d5c4f468d9864 P 082a50743594445aacfab4de6c7d914f: Log is configured to *not* fsync() on all Append() calls
I20260926 16:35:09.090338 27198 tablet_bootstrap.cc:492] T 6bc7565d818741b5984d5c4f468d9864 P 082a50743594445aacfab4de6c7d914f: No bootstrap required, opened a new log
I20260926 16:35:09.090518 27198 ts_tablet_manager.cc:1397] T 6bc7565d818741b5984d5c4f468d9864 P 082a50743594445aacfab4de6c7d914f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260926 16:35:09.091507 27197 raft_consensus.cc:348] T 6bc7565d818741b5984d5c4f468d9864 P c85733551b2d4dd7b984240ad8ab3224 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c85733551b2d4dd7b984240ad8ab3224" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34339 } } peers { permanent_uuid: "082a50743594445aacfab4de6c7d914f" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46177 } } peers { permanent_uuid: "03003d7115c04b9d90c5c8d257aa5a33" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 38387 } }
I20260926 16:35:09.091791 27197 raft_consensus.cc:374] T 6bc7565d818741b5984d5c4f468d9864 P c85733551b2d4dd7b984240ad8ab3224 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:35:09.091838 27197 raft_consensus.cc:729] T 6bc7565d818741b5984d5c4f468d9864 P c85733551b2d4dd7b984240ad8ab3224 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c85733551b2d4dd7b984240ad8ab3224, State: Initialized, Role: FOLLOWER
I20260926 16:35:09.091703 27196 raft_consensus.cc:348] T 6bc7565d818741b5984d5c4f468d9864 P 03003d7115c04b9d90c5c8d257aa5a33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c85733551b2d4dd7b984240ad8ab3224" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34339 } } peers { permanent_uuid: "082a50743594445aacfab4de6c7d914f" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46177 } } peers { permanent_uuid: "03003d7115c04b9d90c5c8d257aa5a33" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 38387 } }
I20260926 16:35:09.091950 27196 raft_consensus.cc:374] T 6bc7565d818741b5984d5c4f468d9864 P 03003d7115c04b9d90c5c8d257aa5a33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:35:09.091991 27196 raft_consensus.cc:729] T 6bc7565d818741b5984d5c4f468d9864 P 03003d7115c04b9d90c5c8d257aa5a33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03003d7115c04b9d90c5c8d257aa5a33, State: Initialized, Role: FOLLOWER
I20260926 16:35:09.092331 27197 consensus_queue.cc:260] T 6bc7565d818741b5984d5c4f468d9864 P c85733551b2d4dd7b984240ad8ab3224 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c85733551b2d4dd7b984240ad8ab3224" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34339 } } peers { permanent_uuid: "082a50743594445aacfab4de6c7d914f" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46177 } } peers { permanent_uuid: "03003d7115c04b9d90c5c8d257aa5a33" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 38387 } }
I20260926 16:35:09.092330 27196 consensus_queue.cc:260] T 6bc7565d818741b5984d5c4f468d9864 P 03003d7115c04b9d90c5c8d257aa5a33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c85733551b2d4dd7b984240ad8ab3224" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34339 } } peers { permanent_uuid: "082a50743594445aacfab4de6c7d914f" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46177 } } peers { permanent_uuid: "03003d7115c04b9d90c5c8d257aa5a33" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 38387 } }
I20260926 16:35:09.093189 27045 heartbeater.cc:503] Master 127.11.8.254:37989 was elected leader, sending a full tablet report...
I20260926 16:35:09.093220 27179 heartbeater.cc:503] Master 127.11.8.254:37989 was elected leader, sending a full tablet report...
I20260926 16:35:09.093312 27196 ts_tablet_manager.cc:1428] T 6bc7565d818741b5984d5c4f468d9864 P 03003d7115c04b9d90c5c8d257aa5a33: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260926 16:35:09.093312 27197 ts_tablet_manager.cc:1428] T 6bc7565d818741b5984d5c4f468d9864 P c85733551b2d4dd7b984240ad8ab3224: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260926 16:35:09.095602 27198 raft_consensus.cc:348] T 6bc7565d818741b5984d5c4f468d9864 P 082a50743594445aacfab4de6c7d914f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c85733551b2d4dd7b984240ad8ab3224" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34339 } } peers { permanent_uuid: "082a50743594445aacfab4de6c7d914f" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46177 } } peers { permanent_uuid: "03003d7115c04b9d90c5c8d257aa5a33" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 38387 } }
I20260926 16:35:09.095817 27198 raft_consensus.cc:374] T 6bc7565d818741b5984d5c4f468d9864 P 082a50743594445aacfab4de6c7d914f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:35:09.095871 27198 raft_consensus.cc:729] T 6bc7565d818741b5984d5c4f468d9864 P 082a50743594445aacfab4de6c7d914f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 082a50743594445aacfab4de6c7d914f, State: Initialized, Role: FOLLOWER
I20260926 16:35:09.096320 27198 consensus_queue.cc:260] T 6bc7565d818741b5984d5c4f468d9864 P 082a50743594445aacfab4de6c7d914f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c85733551b2d4dd7b984240ad8ab3224" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34339 } } peers { permanent_uuid: "082a50743594445aacfab4de6c7d914f" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46177 } } peers { permanent_uuid: "03003d7115c04b9d90c5c8d257aa5a33" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 38387 } }
I20260926 16:35:09.097280 27198 ts_tablet_manager.cc:1428] T 6bc7565d818741b5984d5c4f468d9864 P 082a50743594445aacfab4de6c7d914f: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260926 16:35:09.097636 26911 heartbeater.cc:503] Master 127.11.8.254:37989 was elected leader, sending a full tablet report...
W20260926 16:35:09.272003 27180 tablet.cc:2369] T 6bc7565d818741b5984d5c4f468d9864 P 03003d7115c04b9d90c5c8d257aa5a33: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:35:09.291251 27204 raft_consensus.cc:482] T 6bc7565d818741b5984d5c4f468d9864 P 082a50743594445aacfab4de6c7d914f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:35:09.291458 27204 raft_consensus.cc:504] T 6bc7565d818741b5984d5c4f468d9864 P 082a50743594445aacfab4de6c7d914f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c85733551b2d4dd7b984240ad8ab3224" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34339 } } peers { permanent_uuid: "082a50743594445aacfab4de6c7d914f" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46177 } } peers { permanent_uuid: "03003d7115c04b9d90c5c8d257aa5a33" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 38387 } }
I20260926 16:35:09.292624 27204 leader_election.cc:288] T 6bc7565d818741b5984d5c4f468d9864 P 082a50743594445aacfab4de6c7d914f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c85733551b2d4dd7b984240ad8ab3224 (127.11.8.194:34339), 03003d7115c04b9d90c5c8d257aa5a33 (127.11.8.195:38387)
I20260926 16:35:09.296561 27191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:09.292981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:45665 (local address 127.11.8.194:34339)
0926 16:35:09.293108 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0926 16:35:09.293113 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:35:09.293303 (+ 190us) server_negotiation.cc:407] Connection header received
0926 16:35:09.293479 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:35:09.293487 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:35:09.293530 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:35:09.293612 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:35:09.293943 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:09.294481 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:09.295509 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:09.295977 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:09.296122 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:09.296165 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0926 16:35:09.296359 (+ 194us) server_negotiation.cc:299] Negotiation successful
0926 16:35:09.296435 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 16:35:09.296588 27205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:09.292845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:34339 (local address 127.11.8.193:45665)
0926 16:35:09.293199 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:09.293214 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:35:09.293372 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:35:09.293623 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:35:09.293626 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:35:09.293640 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:35:09.293837 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:35:09.293842 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:35:09.294582 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:35:09.294587 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:35:09.295392 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:35:09.295400 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:35:09.296125 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:35:09.296128 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:35:09.296226 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:09.296248 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 16:35:09.296316 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 16:35:09.296397 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"mutex_wait_us":37,"spinlock_wait_cycles":20864,"thread_start_us":116,"threads_started":1}
I20260926 16:35:09.296898 27195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:09.293062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:33033 (local address 127.11.8.195:38387)
0926 16:35:09.293212 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0926 16:35:09.293217 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:35:09.293352 (+ 135us) server_negotiation.cc:407] Connection header received
0926 16:35:09.293501 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:35:09.293504 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:35:09.293554 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:35:09.293641 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:35:09.293920 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:09.294491 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:09.295569 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:09.296007 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:09.296137 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:09.296186 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 16:35:09.296667 (+ 481us) server_negotiation.cc:299] Negotiation successful
0926 16:35:09.296748 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 16:35:09.297281 27206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:09.292945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:38387 (local address 127.11.8.193:33033)
0926 16:35:09.293261 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:09.293273 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:35:09.293394 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:35:09.293649 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:35:09.293652 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:35:09.293668 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:35:09.293819 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:35:09.293825 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:35:09.294594 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:35:09.294597 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:35:09.295448 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:35:09.295457 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:35:09.296424 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:35:09.296427 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:35:09.296520 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:09.296542 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 16:35:09.297028 (+ 486us) client_negotiation.cc:240] Negotiation successful
0926 16:35:09.297104 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"mutex_wait_us":3,"thread_start_us":103,"threads_started":1}
I20260926 16:35:09.297526 26999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bc7565d818741b5984d5c4f468d9864" candidate_uuid: "082a50743594445aacfab4de6c7d914f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c85733551b2d4dd7b984240ad8ab3224" is_pre_election: true
I20260926 16:35:09.297832 26999 raft_consensus.cc:2454] T 6bc7565d818741b5984d5c4f468d9864 P c85733551b2d4dd7b984240ad8ab3224 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 082a50743594445aacfab4de6c7d914f in term 0.
I20260926 16:35:09.297868 27133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bc7565d818741b5984d5c4f468d9864" candidate_uuid: "082a50743594445aacfab4de6c7d914f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03003d7115c04b9d90c5c8d257aa5a33" is_pre_election: true
I20260926 16:35:09.298190 27133 raft_consensus.cc:2454] T 6bc7565d818741b5984d5c4f468d9864 P 03003d7115c04b9d90c5c8d257aa5a33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 082a50743594445aacfab4de6c7d914f in term 0.
I20260926 16:35:09.298396 26798 leader_election.cc:302] T 6bc7565d818741b5984d5c4f468d9864 P 082a50743594445aacfab4de6c7d914f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 082a50743594445aacfab4de6c7d914f, c85733551b2d4dd7b984240ad8ab3224; no voters:
I20260926 16:35:09.298671 27204 raft_consensus.cc:2781] T 6bc7565d818741b5984d5c4f468d9864 P 082a50743594445aacfab4de6c7d914f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:35:09.298758 27204 raft_consensus.cc:482] T 6bc7565d818741b5984d5c4f468d9864 P 082a50743594445aacfab4de6c7d914f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:35:09.298801 27204 raft_consensus.cc:3037] T 6bc7565d818741b5984d5c4f468d9864 P 082a50743594445aacfab4de6c7d914f [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:35:09.299942 27204 raft_consensus.cc:504] T 6bc7565d818741b5984d5c4f468d9864 P 082a50743594445aacfab4de6c7d914f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c85733551b2d4dd7b984240ad8ab3224" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34339 } } peers { permanent_uuid: "082a50743594445aacfab4de6c7d914f" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46177 } } peers { permanent_uuid: "03003d7115c04b9d90c5c8d257aa5a33" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 38387 } }
I20260926 16:35:09.300297 27204 leader_election.cc:288] T 6bc7565d818741b5984d5c4f468d9864 P 082a50743594445aacfab4de6c7d914f [CANDIDATE]: Term 1 election: Requested vote from peers c85733551b2d4dd7b984240ad8ab3224 (127.11.8.194:34339), 03003d7115c04b9d90c5c8d257aa5a33 (127.11.8.195:38387)
I20260926 16:35:09.300613 26999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bc7565d818741b5984d5c4f468d9864" candidate_uuid: "082a50743594445aacfab4de6c7d914f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c85733551b2d4dd7b984240ad8ab3224"
I20260926 16:35:09.300676 27133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bc7565d818741b5984d5c4f468d9864" candidate_uuid: "082a50743594445aacfab4de6c7d914f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03003d7115c04b9d90c5c8d257aa5a33"
W20260926 16:35:09.300705 27046 tablet.cc:2369] T 6bc7565d818741b5984d5c4f468d9864 P c85733551b2d4dd7b984240ad8ab3224: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:35:09.300735 26999 raft_consensus.cc:3037] T 6bc7565d818741b5984d5c4f468d9864 P c85733551b2d4dd7b984240ad8ab3224 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:35:09.300772 27133 raft_consensus.cc:3037] T 6bc7565d818741b5984d5c4f468d9864 P 03003d7115c04b9d90c5c8d257aa5a33 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:35:09.301996 26999 raft_consensus.cc:2454] T 6bc7565d818741b5984d5c4f468d9864 P c85733551b2d4dd7b984240ad8ab3224 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 082a50743594445aacfab4de6c7d914f in term 1.
I20260926 16:35:09.301996 27133 raft_consensus.cc:2454] T 6bc7565d818741b5984d5c4f468d9864 P 03003d7115c04b9d90c5c8d257aa5a33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 082a50743594445aacfab4de6c7d914f in term 1.
I20260926 16:35:09.302318 26799 leader_election.cc:302] T 6bc7565d818741b5984d5c4f468d9864 P 082a50743594445aacfab4de6c7d914f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03003d7115c04b9d90c5c8d257aa5a33, 082a50743594445aacfab4de6c7d914f; no voters:
I20260926 16:35:09.302485 27204 raft_consensus.cc:2781] T 6bc7565d818741b5984d5c4f468d9864 P 082a50743594445aacfab4de6c7d914f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:35:09.302738 27204 raft_consensus.cc:686] T 6bc7565d818741b5984d5c4f468d9864 P 082a50743594445aacfab4de6c7d914f [term 1 LEADER]: Becoming Leader. State: Replica: 082a50743594445aacfab4de6c7d914f, State: Running, Role: LEADER
I20260926 16:35:09.303078 27204 consensus_queue.cc:237] T 6bc7565d818741b5984d5c4f468d9864 P 082a50743594445aacfab4de6c7d914f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c85733551b2d4dd7b984240ad8ab3224" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34339 } } peers { permanent_uuid: "082a50743594445aacfab4de6c7d914f" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46177 } } peers { permanent_uuid: "03003d7115c04b9d90c5c8d257aa5a33" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 38387 } }
I20260926 16:35:09.306066 26719 catalog_manager.cc:5495] T 6bc7565d818741b5984d5c4f468d9864 P 082a50743594445aacfab4de6c7d914f reported cstate change: term changed from 0 to 1, leader changed from <none> to 082a50743594445aacfab4de6c7d914f (127.11.8.193). New cstate: current_term: 1 leader_uuid: "082a50743594445aacfab4de6c7d914f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c85733551b2d4dd7b984240ad8ab3224" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "082a50743594445aacfab4de6c7d914f" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "03003d7115c04b9d90c5c8d257aa5a33" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 38387 } health_report { overall_health: UNKNOWN } } }
W20260926 16:35:09.319761 26912 tablet.cc:2369] T 6bc7565d818741b5984d5c4f468d9864 P 082a50743594445aacfab4de6c7d914f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:35:09.361999 27194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:09.358716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51120 (local address 127.11.8.193:46177)
0926 16:35:09.358847 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0926 16:35:09.358851 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:35:09.358872 (+ 21us) server_negotiation.cc:407] Connection header received
0926 16:35:09.358984 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:35:09.358986 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:35:09.359040 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:35:09.359131 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:35:09.359350 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:09.359843 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:09.360631 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:09.360832 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:09.360905 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:09.361194 (+ 289us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:35:09.361602 (+ 408us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:35:09.361696 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 16:35:09.361786 (+ 90us) server_negotiation.cc:299] Negotiation successful
0926 16:35:09.361834 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 16:35:09.367895 27133 raft_consensus.cc:1261] T 6bc7565d818741b5984d5c4f468d9864 P 03003d7115c04b9d90c5c8d257aa5a33 [term 1 FOLLOWER]: Refusing update from remote peer 082a50743594445aacfab4de6c7d914f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:35:09.367918 26999 raft_consensus.cc:1261] T 6bc7565d818741b5984d5c4f468d9864 P c85733551b2d4dd7b984240ad8ab3224 [term 1 FOLLOWER]: Refusing update from remote peer 082a50743594445aacfab4de6c7d914f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:35:09.368613 27207 consensus_queue.cc:1035] T 6bc7565d818741b5984d5c4f468d9864 P 082a50743594445aacfab4de6c7d914f [LEADER]: Connected to new peer: Peer: permanent_uuid: "03003d7115c04b9d90c5c8d257aa5a33" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 38387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:35:09.368799 27204 consensus_queue.cc:1035] T 6bc7565d818741b5984d5c4f468d9864 P 082a50743594445aacfab4de6c7d914f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c85733551b2d4dd7b984240ad8ab3224" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:35:09.386473 26720 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:35:09.386638 26720 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:35:09.388854 26720 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c: Sending DeleteTablet for 3 replicas of tablet 6bc7565d818741b5984d5c4f468d9864
I20260926 16:35:09.389595 26845 tablet_service.cc:1514] Processing DeleteTablet for tablet 6bc7565d818741b5984d5c4f468d9864 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:35:09 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:51104
I20260926 16:35:09.389611 26979 tablet_service.cc:1514] Processing DeleteTablet for tablet 6bc7565d818741b5984d5c4f468d9864 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:35:09 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:58452
I20260926 16:35:09.389693 27113 tablet_service.cc:1514] Processing DeleteTablet for tablet 6bc7565d818741b5984d5c4f468d9864 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:35:09 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:37566
I20260926 16:35:09.389838 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 26782
I20260926 16:35:09.390632 27221 tablet_replica.cc:330] T 6bc7565d818741b5984d5c4f468d9864 P c85733551b2d4dd7b984240ad8ab3224: stopping tablet replica
I20260926 16:35:09.390954 27221 raft_consensus.cc:2229] T 6bc7565d818741b5984d5c4f468d9864 P c85733551b2d4dd7b984240ad8ab3224 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:35:09.391252 27221 raft_consensus.cc:2258] T 6bc7565d818741b5984d5c4f468d9864 P c85733551b2d4dd7b984240ad8ab3224 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:35:09.391619 27222 tablet_replica.cc:330] T 6bc7565d818741b5984d5c4f468d9864 P 03003d7115c04b9d90c5c8d257aa5a33: stopping tablet replica
I20260926 16:35:09.391893 27222 raft_consensus.cc:2229] T 6bc7565d818741b5984d5c4f468d9864 P 03003d7115c04b9d90c5c8d257aa5a33 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:35:09.392148 27222 raft_consensus.cc:2258] T 6bc7565d818741b5984d5c4f468d9864 P 03003d7115c04b9d90c5c8d257aa5a33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:35:09.392845 27221 ts_tablet_manager.cc:1905] T 6bc7565d818741b5984d5c4f468d9864 P c85733551b2d4dd7b984240ad8ab3224: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:35:09.393254 27222 ts_tablet_manager.cc:1905] T 6bc7565d818741b5984d5c4f468d9864 P 03003d7115c04b9d90c5c8d257aa5a33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:35:09.395563 27222 ts_tablet_manager.cc:1918] T 6bc7565d818741b5984d5c4f468d9864 P 03003d7115c04b9d90c5c8d257aa5a33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:35:09.395668 27222 log.cc:1196] T 6bc7565d818741b5984d5c4f468d9864 P 03003d7115c04b9d90c5c8d257aa5a33: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/wal/wals/6bc7565d818741b5984d5c4f468d9864
I20260926 16:35:09.396013 27222 ts_tablet_manager.cc:1939] T 6bc7565d818741b5984d5c4f468d9864 P 03003d7115c04b9d90c5c8d257aa5a33: Deleting consensus metadata
I20260926 16:35:09.396382 27221 ts_tablet_manager.cc:1918] T 6bc7565d818741b5984d5c4f468d9864 P c85733551b2d4dd7b984240ad8ab3224: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:35:09.396481 27221 log.cc:1196] T 6bc7565d818741b5984d5c4f468d9864 P c85733551b2d4dd7b984240ad8ab3224: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/wal/wals/6bc7565d818741b5984d5c4f468d9864
I20260926 16:35:09.396799 27221 ts_tablet_manager.cc:1939] T 6bc7565d818741b5984d5c4f468d9864 P c85733551b2d4dd7b984240ad8ab3224: Deleting consensus metadata
I20260926 16:35:09.397258 26705 catalog_manager.cc:4841] TS 03003d7115c04b9d90c5c8d257aa5a33 (127.11.8.195:38387): tablet 6bc7565d818741b5984d5c4f468d9864 (table test-table [id=2b9056629aa74348b4274a812331fda7]) successfully deleted
I20260926 16:35:09.398017 26704 catalog_manager.cc:4841] TS c85733551b2d4dd7b984240ad8ab3224 (127.11.8.194:34339): tablet 6bc7565d818741b5984d5c4f468d9864 (table test-table [id=2b9056629aa74348b4274a812331fda7]) successfully deleted
I20260926 16:35:09.398138 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 26916
W20260926 16:35:09.398161 26705 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.8.193:46177 (error 108)
I20260926 16:35:09.403113 27192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:09.402840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:46177 (local address 127.0.0.1:51128)
0926 16:35:09.402972 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:09.403039 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.193:46177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260926 16:35:09.403407 26705 catalog_manager.cc:4584] TS 082a50743594445aacfab4de6c7d914f (127.11.8.193:46177): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6bc7565d818741b5984d5c4f468d9864: Network error: Client connection negotiation failed: client connection to 127.11.8.193:46177: connect: Connection refused (error 111)
I20260926 16:35:09.404023 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 27050
I20260926 16:35:09.409473 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 26688
I20260926 16:35:29.415863 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:37989
--webserver_interface=127.11.8.254
--webserver_port=37907
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:38229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:37989
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:35:29.525556 27224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:35:29.525830 27224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:35:29.525900 27224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:35:29.529805 27224 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 16:35:29.529882 27224 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 16:35:29.529910 27224 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 16:35:29.529934 27224 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:35:29.529974 27224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:35:29.530011 27224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:35:29.530066 27224 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:35:29.530104 27224 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:35:29.535221 27224 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:38229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:37989
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:37989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=37907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:35:29.536224 27224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:35:29.537214 27224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 16:35:29.550578 27224 server_base.cc:1034] running on GCE node
W20260926 16:35:29.550535 27233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:35:29.550535 27231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:35:29.550544 27230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:35:29.551496 27224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:35:29.552659 27224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:35:29.553850 27224 hybrid_clock.cc:648] HybridClock initialized: now 1790440529553829 us; error 30 us; skew 500 ppm
Sep 26 16:35:29 dist-test-slave-2zvq krb5kdc[26675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440529, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:35:29.561965 27224 init.cc:377] Logged in from keytab as kudu/127.11.8.254@KRBTEST.COM (short username kudu)
I20260926 16:35:29.562935 27224 webserver.cc:460] Webserver started at http://127.11.8.254:37907/ using document root <none> and password file <none>
I20260926 16:35:29.563525 27224 fs_manager.cc:359] Metadata directory not provided
I20260926 16:35:29.563629 27224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:35:29.567471 27224 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260926 16:35:29.569679 27240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:35:29.570931 27224 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:35:29.571079 27224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "d12c7ef7fd4a497bb2d352f4f42c610c"
format_stamp: "Formatted at 2026-09-26 16:35:08 on dist-test-slave-2zvq"
server_key: "bf68e0ad6d7e03cb20519faec7a0e272"
server_key_iv: "4640bd55d0858f7a4575c9e1292d46d9"
server_key_version: "encryptionkey@0"
I20260926 16:35:29.571671 27224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:35:29.611690 27224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:35:29.616019 27224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:35:29.616219 27224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:35:29.626540 27224 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:37989
I20260926 16:35:29.626847 27292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:37989 every 8 connection(s)
I20260926 16:35:29.628155 27224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:35:29.632700 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 27224
I20260926 16:35:29.633023 27293 sys_catalog.cc:263] Verifying existing consensus state
I20260926 16:35:29.633232 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.193:46177
--local_ip_for_outbound_sockets=127.11.8.193
--tserver_master_addrs=127.11.8.254:37989
--webserver_port=38551
--webserver_interface=127.11.8.193
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:38229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 16:35:29.635686 27293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c: Bootstrap starting.
I20260926 16:35:29.654160 27293 log.cc:824] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c: Log is configured to *not* fsync() on all Append() calls
I20260926 16:35:29.662758 27293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 16:35:29.663146 27293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c: Bootstrap complete.
I20260926 16:35:29.667248 27293 raft_consensus.cc:348] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d12c7ef7fd4a497bb2d352f4f42c610c" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37989 } }
I20260926 16:35:29.667671 27293 raft_consensus.cc:729] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d12c7ef7fd4a497bb2d352f4f42c610c, State: Initialized, Role: FOLLOWER
I20260926 16:35:29.668190 27293 consensus_queue.cc:260] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d12c7ef7fd4a497bb2d352f4f42c610c" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37989 } }
I20260926 16:35:29.668357 27293 raft_consensus.cc:388] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:35:29.668437 27293 raft_consensus.cc:482] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:35:29.668533 27293 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [term 1 FOLLOWER]: Advancing to term 2
I20260926 16:35:29.670433 27293 raft_consensus.cc:504] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d12c7ef7fd4a497bb2d352f4f42c610c" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37989 } }
I20260926 16:35:29.670760 27293 leader_election.cc:302] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d12c7ef7fd4a497bb2d352f4f42c610c; no voters:
I20260926 16:35:29.671097 27293 leader_election.cc:288] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 16:35:29.671177 27297 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [term 2 FOLLOWER]: Leader election won for term 2
I20260926 16:35:29.671368 27297 raft_consensus.cc:686] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [term 2 LEADER]: Becoming Leader. State: Replica: d12c7ef7fd4a497bb2d352f4f42c610c, State: Running, Role: LEADER
I20260926 16:35:29.671650 27297 consensus_queue.cc:237] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d12c7ef7fd4a497bb2d352f4f42c610c" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37989 } }
I20260926 16:35:29.672293 27293 sys_catalog.cc:564] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:35:29.672911 27298 sys_catalog.cc:455] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d12c7ef7fd4a497bb2d352f4f42c610c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d12c7ef7fd4a497bb2d352f4f42c610c" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37989 } } }
I20260926 16:35:29.673039 27298 sys_catalog.cc:458] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [sys.catalog]: This master's current role is: LEADER
I20260926 16:35:29.673857 27304 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:35:29.674423 27299 sys_catalog.cc:455] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d12c7ef7fd4a497bb2d352f4f42c610c. Latest consensus state: current_term: 2 leader_uuid: "d12c7ef7fd4a497bb2d352f4f42c610c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d12c7ef7fd4a497bb2d352f4f42c610c" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37989 } } }
I20260926 16:35:29.674516 27299 sys_catalog.cc:458] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [sys.catalog]: This master's current role is: LEADER
I20260926 16:35:29.678560 27304 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:35:29.679288 27304 catalog_manager.cc:1259] Loaded cluster ID: 4bccdb7bda82407f820ba4ef70913ba4
I20260926 16:35:29.679361 27304 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:35:29.680994 27304 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:35:29.681562 27304 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c: Loaded TSK: 0
I20260926 16:35:29.697300 27304 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c: Generated new TSK 1
I20260926 16:35:29.697557 27304 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 16:35:29.773597 27295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:35:29.773949 27295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:35:29.774013 27295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:35:29.779919 27295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:35:29.779999 27295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:35:29.780085 27295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.193
I20260926 16:35:29.787613 27295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:38229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.193:46177
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.193
--webserver_port=38551
--tserver_master_addrs=127.11.8.254:37989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.193
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:35:29.788893 27295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:35:29.790153 27295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:35:29.800985 27323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:35:29.801079 27321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:35:29.801234 27295 server_base.cc:1034] running on GCE node
W20260926 16:35:29.800995 27320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:35:29.801846 27295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:35:29.802513 27295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:35:29.803719 27295 hybrid_clock.cc:648] HybridClock initialized: now 1790440529803688 us; error 31 us; skew 500 ppm
Sep 26 16:35:29 dist-test-slave-2zvq krb5kdc[26675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440529, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:35:29.807056 27295 init.cc:377] Logged in from keytab as kudu/127.11.8.193@KRBTEST.COM (short username kudu)
I20260926 16:35:29.808037 27295 webserver.cc:460] Webserver started at http://127.11.8.193:38551/ using document root <none> and password file <none>
I20260926 16:35:29.808600 27295 fs_manager.cc:359] Metadata directory not provided
I20260926 16:35:29.808705 27295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:35:29.812805 27295 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 16:35:29.815236 27330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:35:29.816473 27295 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260926 16:35:29.816601 27295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/wal
uuid: "082a50743594445aacfab4de6c7d914f"
format_stamp: "Formatted at 2026-09-26 16:35:08 on dist-test-slave-2zvq"
server_key: "04c87e70745a85f80eede664d0e6f031"
server_key_iv: "cba335d4e28f3cc8d729c4887234d341"
server_key_version: "encryptionkey@0"
I20260926 16:35:29.817137 27295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:35:29.836254 27295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:35:29.839310 27295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:35:29.839505 27295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:35:29.840086 27295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:35:29.841228 27337 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 16:35:29.846375 27295 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 16:35:29.846457 27295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260926 16:35:29.846544 27295 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 16:35:29.848220 27295 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 16:35:29.848294 27295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260926 16:35:29.848628 27337 tablet_bootstrap.cc:492] T 6bc7565d818741b5984d5c4f468d9864 P 082a50743594445aacfab4de6c7d914f: Bootstrap starting.
I20260926 16:35:29.858871 27295 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.193:46177
I20260926 16:35:29.859100 27444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.193:46177 every 8 connection(s)
I20260926 16:35:29.859905 27295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
I20260926 16:35:29.861364 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 27295
I20260926 16:35:29.861879 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.194:34339
--local_ip_for_outbound_sockets=127.11.8.194
--tserver_master_addrs=127.11.8.254:37989
--webserver_port=38161
--webserver_interface=127.11.8.194
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:38229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 16:35:29.872519 27337 log.cc:824] T 6bc7565d818741b5984d5c4f468d9864 P 082a50743594445aacfab4de6c7d914f: Log is configured to *not* fsync() on all Append() calls
Sep 26 16:35:29 dist-test-slave-2zvq krb5kdc[26675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440529, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:35:29.880999 27337 tablet_bootstrap.cc:492] T 6bc7565d818741b5984d5c4f468d9864 P 082a50743594445aacfab4de6c7d914f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 16:35:29.881508 27337 tablet_bootstrap.cc:492] T 6bc7565d818741b5984d5c4f468d9864 P 082a50743594445aacfab4de6c7d914f: Bootstrap complete.
I20260926 16:35:29.882608 27448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:29.862056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.193:39675)
0926 16:35:29.863644 (+ 1588us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:29.863707 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0926 16:35:29.864817 (+ 1110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:35:29.866498 (+ 1681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:35:29.866509 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:35:29.867050 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:35:29.867808 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:35:29.867834 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:35:29.869901 (+ 2067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:35:29.869906 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:35:29.871536 (+ 1630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:35:29.871553 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:35:29.873188 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:35:29.873194 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:35:29.873504 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:29.874215 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:35:29.874242 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:35:29.876645 (+ 2403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:35:29.879151 (+ 2506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:35:29.879157 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:35:29.879168 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:35:29.879502 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:35:29.879890 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:35:29.879893 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:35:29.879895 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:35:29.880057 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:35:29.880713 (+ 656us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:35:29.880721 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:35:29.880996 (+ 275us) client_negotiation.cc:769] Sending connection context
0926 16:35:29.881213 (+ 217us) client_negotiation.cc:240] Negotiation successful
0926 16:35:29.881561 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1340,"thread_start_us":143,"threads_started":1}
I20260926 16:35:29.883106 27337 ts_tablet_manager.cc:1397] T 6bc7565d818741b5984d5c4f468d9864 P 082a50743594445aacfab4de6c7d914f: Time spent bootstrapping tablet: real 0.035s user 0.021s sys 0.011s
I20260926 16:35:29.883908 27447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:29.862100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:39675 (local address 127.11.8.254:37989)
0926 16:35:29.864697 (+ 2597us) server_negotiation.cc:206] Beginning negotiation
0926 16:35:29.864715 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 16:35:29.864765 (+ 50us) server_negotiation.cc:407] Connection header received
0926 16:35:29.865249 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:35:29.865277 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:35:29.865664 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:35:29.866022 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:35:29.868080 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:29.869697 (+ 1617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:29.871737 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:29.872696 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:29.872946 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:29.876888 (+ 3942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:35:29.876895 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:35:29.876911 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:35:29.876943 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:35:29.878992 (+ 2049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:35:29.879630 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:35:29.879636 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:35:29.879641 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:35:29.879786 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:35:29.880163 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:35:29.880166 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:35:29.880168 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:35:29.880586 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:35:29.880710 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0926 16:35:29.882515 (+ 1805us) server_negotiation.cc:299] Negotiation successful
0926 16:35:29.883126 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2454,"thread_start_us":100,"threads_started":1}
I20260926 16:35:29.885591 27337 raft_consensus.cc:348] T 6bc7565d818741b5984d5c4f468d9864 P 082a50743594445aacfab4de6c7d914f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c85733551b2d4dd7b984240ad8ab3224" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34339 } } peers { permanent_uuid: "082a50743594445aacfab4de6c7d914f" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46177 } } peers { permanent_uuid: "03003d7115c04b9d90c5c8d257aa5a33" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 38387 } }
I20260926 16:35:29.886119 27337 raft_consensus.cc:729] T 6bc7565d818741b5984d5c4f468d9864 P 082a50743594445aacfab4de6c7d914f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 082a50743594445aacfab4de6c7d914f, State: Initialized, Role: FOLLOWER
I20260926 16:35:29.886137 27445 heartbeater.cc:347] Connected to a master server at 127.11.8.254:37989
I20260926 16:35:29.886406 27445 heartbeater.cc:464] Registering TS with master...
I20260926 16:35:29.886674 27337 consensus_queue.cc:260] T 6bc7565d818741b5984d5c4f468d9864 P 082a50743594445aacfab4de6c7d914f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c85733551b2d4dd7b984240ad8ab3224" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34339 } } peers { permanent_uuid: "082a50743594445aacfab4de6c7d914f" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46177 } } peers { permanent_uuid: "03003d7115c04b9d90c5c8d257aa5a33" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 38387 } }
I20260926 16:35:29.886960 27445 heartbeater.cc:511] Master 127.11.8.254:37989 requested a full tablet report, sending...
I20260926 16:35:29.887615 27337 ts_tablet_manager.cc:1428] T 6bc7565d818741b5984d5c4f468d9864 P 082a50743594445aacfab4de6c7d914f: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260926 16:35:29.889344 27251 ts_manager.cc:194] Registered new tserver with Master: 082a50743594445aacfab4de6c7d914f (127.11.8.193:46177)
I20260926 16:35:29.892647 27251 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.193@KRBTEST.COM'} at 127.11.8.193:39675
I20260926 16:35:29.895013 27445 heartbeater.cc:503] Master 127.11.8.254:37989 was elected leader, sending a full tablet report...
Sep 26 16:35:29 dist-test-slave-2zvq krb5kdc[26675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440529, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for kudu/127.11.8.193@KRBTEST.COM
I20260926 16:35:29.902696 27452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:29.891967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:46177 (local address 127.0.0.1:54622)
0926 16:35:29.892560 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:29.892602 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 16:35:29.892720 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:35:29.893294 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:35:29.893299 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:35:29.893806 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:35:29.893933 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:35:29.893941 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:35:29.895213 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:35:29.895217 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:35:29.896128 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:35:29.896139 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:35:29.896819 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:35:29.896823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:35:29.896927 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:29.897468 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:35:29.897483 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:35:29.899034 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:35:29.900970 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:35:29.900979 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:35:29.900984 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:35:29.901232 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:35:29.901518 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:35:29.901523 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:35:29.901525 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:35:29.901609 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:35:29.902031 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:35:29.902037 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:35:29.902155 (+ 118us) client_negotiation.cc:769] Sending connection context
0926 16:35:29.902350 (+ 195us) client_negotiation.cc:240] Negotiation successful
0926 16:35:29.902528 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":106,"threads_started":1}
I20260926 16:35:29.902734 27453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:29.892081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54622 (local address 127.11.8.193:46177)
0926 16:35:29.892665 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0926 16:35:29.892678 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 16:35:29.892701 (+ 23us) server_negotiation.cc:407] Connection header received
0926 16:35:29.892921 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:35:29.892926 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:35:29.893117 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:35:29.893269 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:35:29.894063 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:29.895092 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:29.896265 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:29.896703 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:29.896788 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:29.899186 (+ 2398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:35:29.899200 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:35:29.899209 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:35:29.899229 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:35:29.900846 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:35:29.901345 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:35:29.901350 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:35:29.901352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:35:29.901428 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:35:29.901700 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:35:29.901706 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:35:29.901710 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:35:29.901921 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:35:29.902055 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0926 16:35:29.902428 (+ 373us) server_negotiation.cc:299] Negotiation successful
0926 16:35:29.902579 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":101,"threads_started":1}
I20260926 16:35:29.903595 27379 tablet_service.cc:1514] Processing DeleteTablet for tablet 6bc7565d818741b5984d5c4f468d9864 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:35:09 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:54622
I20260926 16:35:29.904510 27454 tablet_replica.cc:330] stopping tablet replica
I20260926 16:35:29.904785 27454 raft_consensus.cc:2229] T 6bc7565d818741b5984d5c4f468d9864 P 082a50743594445aacfab4de6c7d914f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:35:29.905009 27454 raft_consensus.cc:2258] T 6bc7565d818741b5984d5c4f468d9864 P 082a50743594445aacfab4de6c7d914f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:35:29.905881 27454 ts_tablet_manager.cc:1905] T 6bc7565d818741b5984d5c4f468d9864 P 082a50743594445aacfab4de6c7d914f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:35:29.909652 27454 ts_tablet_manager.cc:1918] T 6bc7565d818741b5984d5c4f468d9864 P 082a50743594445aacfab4de6c7d914f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:35:29.909765 27454 log.cc:1196] T 6bc7565d818741b5984d5c4f468d9864 P 082a50743594445aacfab4de6c7d914f: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/wal/wals/6bc7565d818741b5984d5c4f468d9864
I20260926 16:35:29.910027 27454 ts_tablet_manager.cc:1939] T 6bc7565d818741b5984d5c4f468d9864 P 082a50743594445aacfab4de6c7d914f: Deleting consensus metadata
I20260926 16:35:29.911787 27242 catalog_manager.cc:4841] TS 082a50743594445aacfab4de6c7d914f (127.11.8.193:46177): tablet 6bc7565d818741b5984d5c4f468d9864 (table test-table [id=2b9056629aa74348b4274a812331fda7]) successfully deleted
W20260926 16:35:29.982056 27449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:35:29.982347 27449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:35:29.982412 27449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:35:29.986334 27449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:35:29.986379 27449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:35:29.986434 27449 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.194
I20260926 16:35:29.991389 27449 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:38229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.194:34339
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.194
--webserver_port=38161
--tserver_master_addrs=127.11.8.254:37989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.194
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:35:29.992272 27449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:35:29.993193 27449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:35:30.002977 27459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:35:30.003031 27449 server_base.cc:1034] running on GCE node
W20260926 16:35:30.003206 27460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:35:30.002996 27462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:35:30.003616 27449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:35:30.004228 27449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:35:30.005419 27449 hybrid_clock.cc:648] HybridClock initialized: now 1790440530005403 us; error 25 us; skew 500 ppm
Sep 26 16:35:30 dist-test-slave-2zvq krb5kdc[26675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440530, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:35:30.008108 27449 init.cc:377] Logged in from keytab as kudu/127.11.8.194@KRBTEST.COM (short username kudu)
I20260926 16:35:30.008913 27449 webserver.cc:460] Webserver started at http://127.11.8.194:38161/ using document root <none> and password file <none>
I20260926 16:35:30.009406 27449 fs_manager.cc:359] Metadata directory not provided
I20260926 16:35:30.009474 27449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:35:30.013142 27449 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:35:30.015066 27469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:35:30.016234 27449 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:35:30.016328 27449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/wal
uuid: "c85733551b2d4dd7b984240ad8ab3224"
format_stamp: "Formatted at 2026-09-26 16:35:08 on dist-test-slave-2zvq"
server_key: "fb9e803b8efcad51e23c812fda8f78a8"
server_key_iv: "d3ebeccfddfb8d1249e0959af884e529"
server_key_version: "encryptionkey@0"
I20260926 16:35:30.016814 27449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:35:30.037978 27449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:35:30.040797 27449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:35:30.040951 27449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:35:30.041471 27449 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:35:30.042307 27449 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:35:30.042359 27449 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:35:30.042399 27449 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:35:30.042420 27449 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:35:30.051401 27449 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.194:34339
I20260926 16:35:30.051427 27582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.194:34339 every 8 connection(s)
I20260926 16:35:30.052353 27449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
I20260926 16:35:30.059660 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 27449
I20260926 16:35:30.060181 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.195:38387
--local_ip_for_outbound_sockets=127.11.8.195
--tserver_master_addrs=127.11.8.254:37989
--webserver_port=38133
--webserver_interface=127.11.8.195
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:38229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 16:35:30 dist-test-slave-2zvq krb5kdc[26675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440530, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:35:30.066462 27447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:30.054120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:53765 (local address 127.11.8.254:37989)
0926 16:35:30.054285 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0926 16:35:30.054290 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:35:30.055568 (+ 1278us) server_negotiation.cc:407] Connection header received
0926 16:35:30.056376 (+ 808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:35:30.056379 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:35:30.056428 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:35:30.056502 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:35:30.057597 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:30.058132 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:30.059299 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:30.059538 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:30.059615 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:30.062506 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:35:30.062510 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:35:30.062514 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:35:30.062546 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:35:30.064251 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:35:30.064758 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:35:30.064764 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:35:30.064768 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:35:30.064876 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:35:30.065210 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:35:30.065216 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:35:30.065220 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:35:30.065476 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:35:30.065584 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 16:35:30.066037 (+ 453us) server_negotiation.cc:299] Negotiation successful
0926 16:35:30.066186 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 16:35:30.067041 27585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:30.054565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.194:53765)
0926 16:35:30.055420 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:30.055461 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 16:35:30.056190 (+ 729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:35:30.056656 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:35:30.056662 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:35:30.056998 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:35:30.057469 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:35:30.057482 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:35:30.058229 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:35:30.058231 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:35:30.059192 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:35:30.059201 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:35:30.059684 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:35:30.059688 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:35:30.059884 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:30.060579 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:35:30.060605 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:35:30.062240 (+ 1635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:35:30.064377 (+ 2137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:35:30.064381 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:35:30.064392 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:35:30.064639 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:35:30.064974 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:35:30.064976 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:35:30.064978 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:35:30.065103 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:35:30.065586 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:35:30.065590 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:35:30.065793 (+ 203us) client_negotiation.cc:769] Sending connection context
0926 16:35:30.065931 (+ 138us) client_negotiation.cc:240] Negotiation successful
0926 16:35:30.066159 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"thread_start_us":510,"threads_started":1}
I20260926 16:35:30.068208 27583 heartbeater.cc:347] Connected to a master server at 127.11.8.254:37989
I20260926 16:35:30.068451 27583 heartbeater.cc:464] Registering TS with master...
I20260926 16:35:30.068907 27583 heartbeater.cc:511] Master 127.11.8.254:37989 requested a full tablet report, sending...
I20260926 16:35:30.069986 27251 ts_manager.cc:194] Registered new tserver with Master: c85733551b2d4dd7b984240ad8ab3224 (127.11.8.194:34339)
I20260926 16:35:30.070606 27251 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.194@KRBTEST.COM'} at 127.11.8.194:53765
W20260926 16:35:30.172443 27586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:35:30.172745 27586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:35:30.172822 27586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:35:30.177139 27586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:35:30.177199 27586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:35:30.177286 27586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.195
I20260926 16:35:30.182473 27586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:38229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.195:38387
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.195
--webserver_port=38133
--tserver_master_addrs=127.11.8.254:37989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27586
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.195
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:35:30.183434 27586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:35:30.184399 27586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:35:30.193642 27594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:35:30.193648 27591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:35:30.193650 27592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:35:30.194047 27586 server_base.cc:1034] running on GCE node
I20260926 16:35:30.194480 27586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:35:30.195108 27586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:35:30.196285 27586 hybrid_clock.cc:648] HybridClock initialized: now 1790440530196275 us; error 36 us; skew 500 ppm
Sep 26 16:35:30 dist-test-slave-2zvq krb5kdc[26675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440530, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:35:30.199111 27586 init.cc:377] Logged in from keytab as kudu/127.11.8.195@KRBTEST.COM (short username kudu)
I20260926 16:35:30.199954 27586 webserver.cc:460] Webserver started at http://127.11.8.195:38133/ using document root <none> and password file <none>
I20260926 16:35:30.200443 27586 fs_manager.cc:359] Metadata directory not provided
I20260926 16:35:30.200510 27586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:35:30.204097 27586 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:35:30.206020 27601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:35:30.207227 27586 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:35:30.207324 27586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/wal
uuid: "03003d7115c04b9d90c5c8d257aa5a33"
format_stamp: "Formatted at 2026-09-26 16:35:08 on dist-test-slave-2zvq"
server_key: "0a5993d3a7529ac1925c75221237d151"
server_key_iv: "3c2d75a4e0bb2641545b2e541fd7353b"
server_key_version: "encryptionkey@0"
I20260926 16:35:30.207801 27586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:35:30.225539 27586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:35:30.228291 27586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:35:30.228439 27586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:35:30.228935 27586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:35:30.229733 27586 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:35:30.229779 27586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:35:30.229822 27586 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:35:30.229840 27586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:35:30.238610 27586 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.195:38387
I20260926 16:35:30.238637 27714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.195:38387 every 8 connection(s)
I20260926 16:35:30.239552 27586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
I20260926 16:35:30.247033 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 27586
Sep 26 16:35:30 dist-test-slave-2zvq krb5kdc[26675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440530, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:35:30.252943 27447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:30.241197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:55217 (local address 127.11.8.254:37989)
0926 16:35:30.241343 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0926 16:35:30.241348 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:35:30.242454 (+ 1106us) server_negotiation.cc:407] Connection header received
0926 16:35:30.243310 (+ 856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:35:30.243313 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:35:30.243408 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:35:30.243498 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:35:30.244580 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:30.245096 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:30.246177 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:30.246426 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:30.246503 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:30.249160 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:35:30.249163 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:35:30.249165 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:35:30.249188 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:35:30.250667 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:35:30.251221 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:35:30.251226 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:35:30.251228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:35:30.251334 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:35:30.251693 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:35:30.251698 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:35:30.251703 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:35:30.251949 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:35:30.252063 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0926 16:35:30.252614 (+ 551us) server_negotiation.cc:299] Negotiation successful
0926 16:35:30.252745 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 16:35:30.253895 27717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:30.241607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.195:55217)
0926 16:35:30.242258 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:30.242362 (+ 104us) client_negotiation.cc:174] Beginning negotiation
0926 16:35:30.243072 (+ 710us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:35:30.243651 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:35:30.243657 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:35:30.243990 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:35:30.244446 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:35:30.244459 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:35:30.245205 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:35:30.245207 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:35:30.246070 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:35:30.246079 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:35:30.246537 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:35:30.246539 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:35:30.246669 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:30.247304 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:35:30.247332 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:35:30.248974 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:35:30.250765 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:35:30.250771 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:35:30.250785 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:35:30.251119 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:35:30.251455 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:35:30.251458 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:35:30.251460 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:35:30.251595 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:35:30.252056 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:35:30.252061 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:35:30.252291 (+ 230us) client_negotiation.cc:769] Sending connection context
0926 16:35:30.252539 (+ 248us) client_negotiation.cc:240] Negotiation successful
0926 16:35:30.252870 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":119,"threads_started":1}
I20260926 16:35:30.255120 27715 heartbeater.cc:347] Connected to a master server at 127.11.8.254:37989
I20260926 16:35:30.255411 27715 heartbeater.cc:464] Registering TS with master...
I20260926 16:35:30.255745 27719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:30.247713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45328 (local address 127.11.8.254:37989)
0926 16:35:30.247985 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0926 16:35:30.247991 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:35:30.248076 (+ 85us) server_negotiation.cc:407] Connection header received
0926 16:35:30.248174 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:35:30.248177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:35:30.248221 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:35:30.248321 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:35:30.249075 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:30.249607 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:30.250529 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:30.250832 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:30.250941 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:30.252339 (+ 1398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:35:30.252342 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:35:30.252346 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:35:30.252368 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:35:30.253991 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:35:30.254496 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:35:30.254500 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:35:30.254502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:35:30.254587 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:35:30.254889 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:35:30.254892 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:35:30.254894 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:35:30.255081 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:35:30.255298 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0926 16:35:30.255443 (+ 145us) server_negotiation.cc:299] Negotiation successful
0926 16:35:30.255584 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":91,"threads_started":1}
I20260926 16:35:30.255975 27715 heartbeater.cc:511] Master 127.11.8.254:37989 requested a full tablet report, sending...
I20260926 16:35:30.257018 27257 ts_manager.cc:194] Registered new tserver with Master: 03003d7115c04b9d90c5c8d257aa5a33 (127.11.8.195:38387)
I20260926 16:35:30.257649 27257 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.195@KRBTEST.COM'} at 127.11.8.195:55217
I20260926 16:35:30.269198 11299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 16:35:30.273337 27719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:30.269931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45334 (local address 127.11.8.254:37989)
0926 16:35:30.270051 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0926 16:35:30.270054 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:35:30.270236 (+ 182us) server_negotiation.cc:407] Connection header received
0926 16:35:30.270412 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:35:30.270415 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:35:30.270506 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:35:30.270591 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:35:30.270796 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:30.271286 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:30.272176 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:30.272413 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:30.272490 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:30.272742 (+ 252us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:35:30.272899 (+ 157us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0926 16:35:30.273167 (+ 268us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45334: token expired
Metrics: {"server-negotiator.queue_time_us":47}
W20260926 16:35:30.273394 27719 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45334: token expired
W20260926 16:35:30.273686 27185 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.11.8.254:37989)
I20260926 16:35:30.280750 27719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:30.274055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45346 (local address 127.11.8.254:37989)
0926 16:35:30.274159 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0926 16:35:30.274162 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:35:30.274189 (+ 27us) server_negotiation.cc:407] Connection header received
0926 16:35:30.274318 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:35:30.274321 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:35:30.274363 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:35:30.274433 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:35:30.275101 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:30.275607 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:30.276428 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:30.276620 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:30.276693 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:30.277964 (+ 1271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:35:30.277967 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:35:30.277969 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:35:30.277999 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:35:30.279429 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:35:30.279799 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:35:30.279802 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:35:30.279803 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:35:30.279878 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:35:30.280102 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:35:30.280105 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:35:30.280112 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:35:30.280280 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:35:30.280383 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0926 16:35:30.280536 (+ 153us) server_negotiation.cc:299] Negotiation successful
0926 16:35:30.280621 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 16:35:30.283154 27257 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45346:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:35:30.286365 27257 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:35:30.294373 27379 tablet_service.cc:1467] Processing CreateTablet for tablet 855a4d10b4e342e9b4eefdae4c493934 (DEFAULT_TABLE table=test-table [id=bf6b7d9b4bb04d9db803bdb0d51693de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:35:30.295071 27379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 855a4d10b4e342e9b4eefdae4c493934. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:35:30.297904 27337 tablet_bootstrap.cc:492] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f: Bootstrap starting.
I20260926 16:35:30.298002 27452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:30.293044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:38387 (local address 127.0.0.1:49944)
0926 16:35:30.293163 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:30.293180 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 16:35:30.293258 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:35:30.294240 (+ 982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:35:30.294243 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:35:30.294260 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:35:30.294397 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:35:30.294403 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:35:30.295505 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:35:30.295510 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:35:30.296527 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:35:30.296538 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:35:30.297578 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:35:30.297581 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:35:30.297672 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:30.297714 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 16:35:30.297781 (+ 67us) client_negotiation.cc:240] Negotiation successful
0926 16:35:30.297869 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260926 16:35:30.298530 27725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:30.293227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49944 (local address 127.11.8.195:38387)
0926 16:35:30.293771 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0926 16:35:30.293783 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 16:35:30.293808 (+ 25us) server_negotiation.cc:407] Connection header received
0926 16:35:30.293888 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:35:30.293895 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:35:30.294089 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:35:30.294261 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:35:30.294494 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:30.295393 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:30.296664 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:30.297460 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:30.297579 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:30.297724 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0926 16:35:30.297831 (+ 107us) server_negotiation.cc:299] Negotiation successful
0926 16:35:30.297922 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":198,"threads_started":1}
I20260926 16:35:30.298712 27724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:30.293301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:34339 (local address 127.0.0.1:40126)
0926 16:35:30.293611 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:30.293626 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:35:30.293716 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:35:30.294453 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:35:30.294456 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:35:30.294474 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:35:30.294565 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:35:30.294571 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:35:30.296032 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:35:30.296035 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:35:30.297155 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:35:30.297164 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:35:30.298241 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:35:30.298247 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:35:30.298376 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:30.298402 (+ 26us) client_negotiation.cc:769] Sending connection context
0926 16:35:30.298472 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 16:35:30.298543 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":136,"threads_started":1}
I20260926 16:35:30.298830 27337 tablet_bootstrap.cc:654] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f: Neither blocks nor log segments found. Creating new log.
I20260926 16:35:30.299093 27726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:30.293547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40126 (local address 127.11.8.194:34339)
0926 16:35:30.293964 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0926 16:35:30.293975 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 16:35:30.293998 (+ 23us) server_negotiation.cc:407] Connection header received
0926 16:35:30.294072 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:35:30.294077 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:35:30.294294 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:35:30.294707 (+ 413us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:35:30.294896 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:30.295911 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:30.297292 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:30.298115 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:30.298224 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:30.298393 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0926 16:35:30.298666 (+ 273us) server_negotiation.cc:299] Negotiation successful
0926 16:35:30.298941 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":128,"threads_started":1}
I20260926 16:35:30.301656 27649 tablet_service.cc:1467] Processing CreateTablet for tablet 855a4d10b4e342e9b4eefdae4c493934 (DEFAULT_TABLE table=test-table [id=bf6b7d9b4bb04d9db803bdb0d51693de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:35:30.301654 27517 tablet_service.cc:1467] Processing CreateTablet for tablet 855a4d10b4e342e9b4eefdae4c493934 (DEFAULT_TABLE table=test-table [id=bf6b7d9b4bb04d9db803bdb0d51693de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:35:30.302578 27337 tablet_bootstrap.cc:492] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f: No bootstrap required, opened a new log
I20260926 16:35:30.302637 27517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 855a4d10b4e342e9b4eefdae4c493934. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:35:30.302652 27649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 855a4d10b4e342e9b4eefdae4c493934. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:35:30.302664 27337 ts_tablet_manager.cc:1397] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20260926 16:35:30.303129 27337 raft_consensus.cc:348] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03003d7115c04b9d90c5c8d257aa5a33" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 38387 } } peers { permanent_uuid: "c85733551b2d4dd7b984240ad8ab3224" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34339 } } peers { permanent_uuid: "082a50743594445aacfab4de6c7d914f" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46177 } }
I20260926 16:35:30.303248 27337 raft_consensus.cc:374] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:35:30.303292 27337 raft_consensus.cc:729] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 082a50743594445aacfab4de6c7d914f, State: Initialized, Role: FOLLOWER
I20260926 16:35:30.303416 27337 consensus_queue.cc:260] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03003d7115c04b9d90c5c8d257aa5a33" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 38387 } } peers { permanent_uuid: "c85733551b2d4dd7b984240ad8ab3224" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34339 } } peers { permanent_uuid: "082a50743594445aacfab4de6c7d914f" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46177 } }
I20260926 16:35:30.303678 27337 ts_tablet_manager.cc:1428] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260926 16:35:30.309981 27729 tablet_bootstrap.cc:492] T 855a4d10b4e342e9b4eefdae4c493934 P 03003d7115c04b9d90c5c8d257aa5a33: Bootstrap starting.
I20260926 16:35:30.310531 27728 tablet_bootstrap.cc:492] T 855a4d10b4e342e9b4eefdae4c493934 P c85733551b2d4dd7b984240ad8ab3224: Bootstrap starting.
I20260926 16:35:30.312892 27728 tablet_bootstrap.cc:654] T 855a4d10b4e342e9b4eefdae4c493934 P c85733551b2d4dd7b984240ad8ab3224: Neither blocks nor log segments found. Creating new log.
I20260926 16:35:30.312896 27729 tablet_bootstrap.cc:654] T 855a4d10b4e342e9b4eefdae4c493934 P 03003d7115c04b9d90c5c8d257aa5a33: Neither blocks nor log segments found. Creating new log.
I20260926 16:35:30.313613 27728 log.cc:824] T 855a4d10b4e342e9b4eefdae4c493934 P c85733551b2d4dd7b984240ad8ab3224: Log is configured to *not* fsync() on all Append() calls
I20260926 16:35:30.313613 27729 log.cc:824] T 855a4d10b4e342e9b4eefdae4c493934 P 03003d7115c04b9d90c5c8d257aa5a33: Log is configured to *not* fsync() on all Append() calls
I20260926 16:35:30.315384 27728 tablet_bootstrap.cc:492] T 855a4d10b4e342e9b4eefdae4c493934 P c85733551b2d4dd7b984240ad8ab3224: No bootstrap required, opened a new log
I20260926 16:35:30.315586 27728 ts_tablet_manager.cc:1397] T 855a4d10b4e342e9b4eefdae4c493934 P c85733551b2d4dd7b984240ad8ab3224: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260926 16:35:30.318068 27729 tablet_bootstrap.cc:492] T 855a4d10b4e342e9b4eefdae4c493934 P 03003d7115c04b9d90c5c8d257aa5a33: No bootstrap required, opened a new log
I20260926 16:35:30.318256 27729 ts_tablet_manager.cc:1397] T 855a4d10b4e342e9b4eefdae4c493934 P 03003d7115c04b9d90c5c8d257aa5a33: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260926 16:35:30.318902 27728 raft_consensus.cc:348] T 855a4d10b4e342e9b4eefdae4c493934 P c85733551b2d4dd7b984240ad8ab3224 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03003d7115c04b9d90c5c8d257aa5a33" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 38387 } } peers { permanent_uuid: "c85733551b2d4dd7b984240ad8ab3224" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34339 } } peers { permanent_uuid: "082a50743594445aacfab4de6c7d914f" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46177 } }
I20260926 16:35:30.319121 27728 raft_consensus.cc:374] T 855a4d10b4e342e9b4eefdae4c493934 P c85733551b2d4dd7b984240ad8ab3224 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:35:30.319191 27728 raft_consensus.cc:729] T 855a4d10b4e342e9b4eefdae4c493934 P c85733551b2d4dd7b984240ad8ab3224 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c85733551b2d4dd7b984240ad8ab3224, State: Initialized, Role: FOLLOWER
I20260926 16:35:30.319597 27728 consensus_queue.cc:260] T 855a4d10b4e342e9b4eefdae4c493934 P c85733551b2d4dd7b984240ad8ab3224 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03003d7115c04b9d90c5c8d257aa5a33" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 38387 } } peers { permanent_uuid: "c85733551b2d4dd7b984240ad8ab3224" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34339 } } peers { permanent_uuid: "082a50743594445aacfab4de6c7d914f" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46177 } }
I20260926 16:35:30.320271 27583 heartbeater.cc:503] Master 127.11.8.254:37989 was elected leader, sending a full tablet report...
I20260926 16:35:30.320677 27728 ts_tablet_manager.cc:1428] T 855a4d10b4e342e9b4eefdae4c493934 P c85733551b2d4dd7b984240ad8ab3224: Time spent starting tablet: real 0.005s user 0.000s sys 0.006s
I20260926 16:35:30.322140 27729 raft_consensus.cc:348] T 855a4d10b4e342e9b4eefdae4c493934 P 03003d7115c04b9d90c5c8d257aa5a33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03003d7115c04b9d90c5c8d257aa5a33" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 38387 } } peers { permanent_uuid: "c85733551b2d4dd7b984240ad8ab3224" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34339 } } peers { permanent_uuid: "082a50743594445aacfab4de6c7d914f" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46177 } }
I20260926 16:35:30.322449 27729 raft_consensus.cc:374] T 855a4d10b4e342e9b4eefdae4c493934 P 03003d7115c04b9d90c5c8d257aa5a33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:35:30.322532 27729 raft_consensus.cc:729] T 855a4d10b4e342e9b4eefdae4c493934 P 03003d7115c04b9d90c5c8d257aa5a33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03003d7115c04b9d90c5c8d257aa5a33, State: Initialized, Role: FOLLOWER
I20260926 16:35:30.322958 27729 consensus_queue.cc:260] T 855a4d10b4e342e9b4eefdae4c493934 P 03003d7115c04b9d90c5c8d257aa5a33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03003d7115c04b9d90c5c8d257aa5a33" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 38387 } } peers { permanent_uuid: "c85733551b2d4dd7b984240ad8ab3224" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34339 } } peers { permanent_uuid: "082a50743594445aacfab4de6c7d914f" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46177 } }
I20260926 16:35:30.323668 27715 heartbeater.cc:503] Master 127.11.8.254:37989 was elected leader, sending a full tablet report...
I20260926 16:35:30.323956 27729 ts_tablet_manager.cc:1428] T 855a4d10b4e342e9b4eefdae4c493934 P 03003d7115c04b9d90c5c8d257aa5a33: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260926 16:35:30.361382 27446 tablet.cc:2369] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:35:30.369077 27451 raft_consensus.cc:482] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:35:30.369272 27451 raft_consensus.cc:504] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03003d7115c04b9d90c5c8d257aa5a33" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 38387 } } peers { permanent_uuid: "c85733551b2d4dd7b984240ad8ab3224" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34339 } } peers { permanent_uuid: "082a50743594445aacfab4de6c7d914f" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46177 } }
I20260926 16:35:30.370527 27451 leader_election.cc:288] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03003d7115c04b9d90c5c8d257aa5a33 (127.11.8.195:38387), c85733551b2d4dd7b984240ad8ab3224 (127.11.8.194:34339)
I20260926 16:35:30.374701 27448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:30.370698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:38387 (local address 127.11.8.193:55727)
0926 16:35:30.370892 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:30.370909 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 16:35:30.371015 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:35:30.371437 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:35:30.371439 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:35:30.371457 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:35:30.371587 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:35:30.371594 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:35:30.372486 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:35:30.372493 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:35:30.373468 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:35:30.373482 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:35:30.374176 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:35:30.374179 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:35:30.374342 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:30.374391 (+ 49us) client_negotiation.cc:769] Sending connection context
0926 16:35:30.374485 (+ 94us) client_negotiation.cc:240] Negotiation successful
0926 16:35:30.374565 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":8}
I20260926 16:35:30.374699 27726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:30.371185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:36687 (local address 127.11.8.194:34339)
0926 16:35:30.371369 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 16:35:30.371373 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:35:30.371388 (+ 15us) server_negotiation.cc:407] Connection header received
0926 16:35:30.371455 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:35:30.371457 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:35:30.371501 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:35:30.371576 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:35:30.371867 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:30.372416 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:30.373641 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:30.374029 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:30.374157 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:30.374209 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 16:35:30.374506 (+ 297us) server_negotiation.cc:299] Negotiation successful
0926 16:35:30.374586 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260926 16:35:30.374720 27725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:30.370814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:55727 (local address 127.11.8.195:38387)
0926 16:35:30.370946 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0926 16:35:30.370952 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:35:30.370968 (+ 16us) server_negotiation.cc:407] Connection header received
0926 16:35:30.371177 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:35:30.371181 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:35:30.371237 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:35:30.371308 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:35:30.371741 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:30.372376 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:30.373608 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:30.374034 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:30.374177 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:30.374229 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 16:35:30.374501 (+ 272us) server_negotiation.cc:299] Negotiation successful
0926 16:35:30.374572 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 16:35:30.374722 27734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:30.370811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:34339 (local address 127.11.8.193:36687)
0926 16:35:30.371253 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:30.371268 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:35:30.371368 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:35:30.371590 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:35:30.371593 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:35:30.371606 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:35:30.371750 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:35:30.371757 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:35:30.372542 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:35:30.372544 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:35:30.373489 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:35:30.373503 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:35:30.374176 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:35:30.374180 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:35:30.374366 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:30.374392 (+ 26us) client_negotiation.cc:769] Sending connection context
0926 16:35:30.374479 (+ 87us) client_negotiation.cc:240] Negotiation successful
0926 16:35:30.374553 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"mutex_wait_us":88,"thread_start_us":93,"threads_started":1}
I20260926 16:35:30.375404 27669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855a4d10b4e342e9b4eefdae4c493934" candidate_uuid: "082a50743594445aacfab4de6c7d914f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03003d7115c04b9d90c5c8d257aa5a33" is_pre_election: true
I20260926 16:35:30.375541 27537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855a4d10b4e342e9b4eefdae4c493934" candidate_uuid: "082a50743594445aacfab4de6c7d914f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c85733551b2d4dd7b984240ad8ab3224" is_pre_election: true
I20260926 16:35:30.375759 27669 raft_consensus.cc:2454] T 855a4d10b4e342e9b4eefdae4c493934 P 03003d7115c04b9d90c5c8d257aa5a33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 082a50743594445aacfab4de6c7d914f in term 0.
I20260926 16:35:30.375855 27537 raft_consensus.cc:2454] T 855a4d10b4e342e9b4eefdae4c493934 P c85733551b2d4dd7b984240ad8ab3224 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 082a50743594445aacfab4de6c7d914f in term 0.
I20260926 16:35:30.376338 27331 leader_election.cc:302] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 082a50743594445aacfab4de6c7d914f, c85733551b2d4dd7b984240ad8ab3224; no voters:
I20260926 16:35:30.376654 27451 raft_consensus.cc:2781] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:35:30.376740 27451 raft_consensus.cc:482] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:35:30.376784 27451 raft_consensus.cc:3037] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:35:30.377810 27451 raft_consensus.cc:504] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03003d7115c04b9d90c5c8d257aa5a33" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 38387 } } peers { permanent_uuid: "c85733551b2d4dd7b984240ad8ab3224" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34339 } } peers { permanent_uuid: "082a50743594445aacfab4de6c7d914f" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46177 } }
I20260926 16:35:30.378132 27451 leader_election.cc:288] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [CANDIDATE]: Term 1 election: Requested vote from peers 03003d7115c04b9d90c5c8d257aa5a33 (127.11.8.195:38387), c85733551b2d4dd7b984240ad8ab3224 (127.11.8.194:34339)
I20260926 16:35:30.378518 27669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855a4d10b4e342e9b4eefdae4c493934" candidate_uuid: "082a50743594445aacfab4de6c7d914f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03003d7115c04b9d90c5c8d257aa5a33"
I20260926 16:35:30.378569 27537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855a4d10b4e342e9b4eefdae4c493934" candidate_uuid: "082a50743594445aacfab4de6c7d914f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c85733551b2d4dd7b984240ad8ab3224"
I20260926 16:35:30.378623 27669 raft_consensus.cc:3037] T 855a4d10b4e342e9b4eefdae4c493934 P 03003d7115c04b9d90c5c8d257aa5a33 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:35:30.378722 27537 raft_consensus.cc:3037] T 855a4d10b4e342e9b4eefdae4c493934 P c85733551b2d4dd7b984240ad8ab3224 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:35:30.379647 27669 raft_consensus.cc:2454] T 855a4d10b4e342e9b4eefdae4c493934 P 03003d7115c04b9d90c5c8d257aa5a33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 082a50743594445aacfab4de6c7d914f in term 1.
I20260926 16:35:30.379848 27537 raft_consensus.cc:2454] T 855a4d10b4e342e9b4eefdae4c493934 P c85733551b2d4dd7b984240ad8ab3224 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 082a50743594445aacfab4de6c7d914f in term 1.
I20260926 16:35:30.380014 27332 leader_election.cc:302] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03003d7115c04b9d90c5c8d257aa5a33, 082a50743594445aacfab4de6c7d914f; no voters:
I20260926 16:35:30.380194 27451 raft_consensus.cc:2781] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:35:30.380437 27451 raft_consensus.cc:686] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [term 1 LEADER]: Becoming Leader. State: Replica: 082a50743594445aacfab4de6c7d914f, State: Running, Role: LEADER
I20260926 16:35:30.380688 27451 consensus_queue.cc:237] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03003d7115c04b9d90c5c8d257aa5a33" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 38387 } } peers { permanent_uuid: "c85733551b2d4dd7b984240ad8ab3224" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34339 } } peers { permanent_uuid: "082a50743594445aacfab4de6c7d914f" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46177 } }
I20260926 16:35:30.383531 27257 catalog_manager.cc:5495] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f reported cstate change: term changed from 0 to 1, leader changed from <none> to 082a50743594445aacfab4de6c7d914f (127.11.8.193). New cstate: current_term: 1 leader_uuid: "082a50743594445aacfab4de6c7d914f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03003d7115c04b9d90c5c8d257aa5a33" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 38387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c85733551b2d4dd7b984240ad8ab3224" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "082a50743594445aacfab4de6c7d914f" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46177 } health_report { overall_health: HEALTHY } } }
I20260926 16:35:30.397811 27453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:30.394125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54634 (local address 127.11.8.193:46177)
0926 16:35:30.394253 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0926 16:35:30.394256 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:35:30.394327 (+ 71us) server_negotiation.cc:407] Connection header received
0926 16:35:30.394427 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:35:30.394429 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:35:30.394481 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:35:30.394566 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:35:30.394848 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:30.395381 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:30.396371 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:30.396594 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:30.396667 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:30.396938 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:35:30.397387 (+ 449us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:35:30.397503 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0926 16:35:30.397649 (+ 146us) server_negotiation.cc:299] Negotiation successful
0926 16:35:30.397704 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 16:35:30.401830 27669 raft_consensus.cc:1261] T 855a4d10b4e342e9b4eefdae4c493934 P 03003d7115c04b9d90c5c8d257aa5a33 [term 1 FOLLOWER]: Refusing update from remote peer 082a50743594445aacfab4de6c7d914f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:35:30.401831 27537 raft_consensus.cc:1261] T 855a4d10b4e342e9b4eefdae4c493934 P c85733551b2d4dd7b984240ad8ab3224 [term 1 FOLLOWER]: Refusing update from remote peer 082a50743594445aacfab4de6c7d914f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:35:30.402554 27451 consensus_queue.cc:1035] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c85733551b2d4dd7b984240ad8ab3224" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:35:30.402792 27735 consensus_queue.cc:1035] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [LEADER]: Connected to new peer: Peer: permanent_uuid: "03003d7115c04b9d90c5c8d257aa5a33" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 38387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:35:30.413254 27738 mvcc.cc:205] Tried to move back new op lower bound from 7333644412519239680 to 7333644412441780224. Current Snapshot: MvccSnapshot[applied={T|T < 7333644412519239680}]
I20260926 16:35:30.415025 27739 mvcc.cc:205] Tried to move back new op lower bound from 7333644412519239680 to 7333644412441780224. Current Snapshot: MvccSnapshot[applied={T|T < 7333644412519239680 or (T in {7333644412519239680})}]
I20260926 16:35:30.417761 27257 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45346:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:35:30.417896 27257 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45346:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:35:30.420321 27257 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c: Sending DeleteTablet for 3 replicas of tablet 855a4d10b4e342e9b4eefdae4c493934
I20260926 16:35:30.420955 27649 tablet_service.cc:1514] Processing DeleteTablet for tablet 855a4d10b4e342e9b4eefdae4c493934 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:35:30 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:49944
I20260926 16:35:30.421077 27517 tablet_service.cc:1514] Processing DeleteTablet for tablet 855a4d10b4e342e9b4eefdae4c493934 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:35:30 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:40126
I20260926 16:35:30.421556 27748 tablet_replica.cc:330] T 855a4d10b4e342e9b4eefdae4c493934 P c85733551b2d4dd7b984240ad8ab3224: stopping tablet replica
I20260926 16:35:30.421698 27379 tablet_service.cc:1514] Processing DeleteTablet for tablet 855a4d10b4e342e9b4eefdae4c493934 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:35:30 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:54622
I20260926 16:35:30.421854 27748 raft_consensus.cc:2229] T 855a4d10b4e342e9b4eefdae4c493934 P c85733551b2d4dd7b984240ad8ab3224 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:35:30.421967 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 27295
I20260926 16:35:30.422138 27748 raft_consensus.cc:2258] T 855a4d10b4e342e9b4eefdae4c493934 P c85733551b2d4dd7b984240ad8ab3224 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:35:30.423789 27748 ts_tablet_manager.cc:1905] T 855a4d10b4e342e9b4eefdae4c493934 P c85733551b2d4dd7b984240ad8ab3224: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:35:30.426483 27747 tablet_replica.cc:330] T 855a4d10b4e342e9b4eefdae4c493934 P 03003d7115c04b9d90c5c8d257aa5a33: stopping tablet replica
I20260926 16:35:30.426753 27747 raft_consensus.cc:2229] T 855a4d10b4e342e9b4eefdae4c493934 P 03003d7115c04b9d90c5c8d257aa5a33 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:35:30.427018 27747 raft_consensus.cc:2258] T 855a4d10b4e342e9b4eefdae4c493934 P 03003d7115c04b9d90c5c8d257aa5a33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:35:30.427196 27748 ts_tablet_manager.cc:1918] T 855a4d10b4e342e9b4eefdae4c493934 P c85733551b2d4dd7b984240ad8ab3224: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:35:30.427304 27748 log.cc:1196] T 855a4d10b4e342e9b4eefdae4c493934 P c85733551b2d4dd7b984240ad8ab3224: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/wal/wals/855a4d10b4e342e9b4eefdae4c493934
I20260926 16:35:30.427637 27748 ts_tablet_manager.cc:1939] T 855a4d10b4e342e9b4eefdae4c493934 P c85733551b2d4dd7b984240ad8ab3224: Deleting consensus metadata
I20260926 16:35:30.428380 27747 ts_tablet_manager.cc:1905] T 855a4d10b4e342e9b4eefdae4c493934 P 03003d7115c04b9d90c5c8d257aa5a33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:35:30.428812 27241 catalog_manager.cc:4841] TS c85733551b2d4dd7b984240ad8ab3224 (127.11.8.194:34339): tablet 855a4d10b4e342e9b4eefdae4c493934 (table test-table [id=bf6b7d9b4bb04d9db803bdb0d51693de]) successfully deleted
I20260926 16:35:30.430541 27747 ts_tablet_manager.cc:1918] T 855a4d10b4e342e9b4eefdae4c493934 P 03003d7115c04b9d90c5c8d257aa5a33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:35:30.430634 27747 log.cc:1196] T 855a4d10b4e342e9b4eefdae4c493934 P 03003d7115c04b9d90c5c8d257aa5a33: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/wal/wals/855a4d10b4e342e9b4eefdae4c493934
W20260926 16:35:30.430631 27242 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.8.193:46177 (error 108)
I20260926 16:35:30.431015 27747 ts_tablet_manager.cc:1939] T 855a4d10b4e342e9b4eefdae4c493934 P 03003d7115c04b9d90c5c8d257aa5a33: Deleting consensus metadata
I20260926 16:35:30.431392 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 27449
I20260926 16:35:30.432814 27242 catalog_manager.cc:4841] TS 03003d7115c04b9d90c5c8d257aa5a33 (127.11.8.195:38387): tablet 855a4d10b4e342e9b4eefdae4c493934 (table test-table [id=bf6b7d9b4bb04d9db803bdb0d51693de]) successfully deleted
I20260926 16:35:30.433179 27724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:30.431371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:46177 (local address 127.0.0.1:54644)
0926 16:35:30.433000 (+ 1629us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:30.433081 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.193:46177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260926 16:35:30.433336 27242 catalog_manager.cc:4584] TS 082a50743594445aacfab4de6c7d914f (127.11.8.193:46177): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 855a4d10b4e342e9b4eefdae4c493934: Network error: Client connection negotiation failed: client connection to 127.11.8.193:46177: connect: Connection refused (error 111)
I20260926 16:35:30.437292 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 27586
I20260926 16:35:30.442515 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 27224
I20260926 16:35:50.448722 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:37989
--webserver_interface=127.11.8.254
--webserver_port=37907
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:38229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:37989
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:35:50.558379 27751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:35:50.558632 27751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:35:50.558673 27751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:35:50.562837 27751 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 16:35:50.562889 27751 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 16:35:50.562904 27751 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 16:35:50.562918 27751 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:35:50.562937 27751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:35:50.562990 27751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:35:50.563017 27751 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:35:50.563055 27751 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:35:50.568639 27751 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:38229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:37989
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:37989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=37907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:35:50.569639 27751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:35:50.570700 27751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:35:50.579391 27759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:35:50.579404 27756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:35:50.579461 27751 server_base.cc:1034] running on GCE node
W20260926 16:35:50.579437 27757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:35:50.580245 27751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:35:50.581373 27751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:35:50.582537 27751 hybrid_clock.cc:648] HybridClock initialized: now 1790440550582525 us; error 29 us; skew 500 ppm
Sep 26 16:35:50 dist-test-slave-2zvq krb5kdc[26675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440550, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:35:50.585325 27751 init.cc:377] Logged in from keytab as kudu/127.11.8.254@KRBTEST.COM (short username kudu)
I20260926 16:35:50.586668 27751 webserver.cc:460] Webserver started at http://127.11.8.254:37907/ using document root <none> and password file <none>
I20260926 16:35:50.587371 27751 fs_manager.cc:359] Metadata directory not provided
I20260926 16:35:50.587472 27751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:35:50.591557 27751 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 16:35:50.593796 27766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:35:50.595093 27751 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:35:50.595238 27751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "d12c7ef7fd4a497bb2d352f4f42c610c"
format_stamp: "Formatted at 2026-09-26 16:35:08 on dist-test-slave-2zvq"
server_key: "bf68e0ad6d7e03cb20519faec7a0e272"
server_key_iv: "4640bd55d0858f7a4575c9e1292d46d9"
server_key_version: "encryptionkey@0"
I20260926 16:35:50.595790 27751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:35:50.611882 27751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:35:50.615052 27751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:35:50.615264 27751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:35:50.622304 27751 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:37989
I20260926 16:35:50.622591 27818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:37989 every 8 connection(s)
I20260926 16:35:50.623466 27751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:35:50.625140 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 27751
I20260926 16:35:50.625661 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.193:46177
--local_ip_for_outbound_sockets=127.11.8.193
--tserver_master_addrs=127.11.8.254:37989
--webserver_port=38551
--webserver_interface=127.11.8.193
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:38229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 16:35:50.628232 27819 sys_catalog.cc:263] Verifying existing consensus state
I20260926 16:35:50.629894 27819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c: Bootstrap starting.
I20260926 16:35:50.647399 27819 log.cc:824] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c: Log is configured to *not* fsync() on all Append() calls
I20260926 16:35:50.658847 27819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 16:35:50.659294 27819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c: Bootstrap complete.
I20260926 16:35:50.663502 27819 raft_consensus.cc:348] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d12c7ef7fd4a497bb2d352f4f42c610c" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37989 } }
I20260926 16:35:50.663890 27819 raft_consensus.cc:729] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d12c7ef7fd4a497bb2d352f4f42c610c, State: Initialized, Role: FOLLOWER
I20260926 16:35:50.664366 27819 consensus_queue.cc:260] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d12c7ef7fd4a497bb2d352f4f42c610c" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37989 } }
I20260926 16:35:50.664479 27819 raft_consensus.cc:388] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:35:50.664562 27819 raft_consensus.cc:482] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:35:50.664677 27819 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [term 2 FOLLOWER]: Advancing to term 3
I20260926 16:35:50.666591 27819 raft_consensus.cc:504] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d12c7ef7fd4a497bb2d352f4f42c610c" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37989 } }
I20260926 16:35:50.666917 27819 leader_election.cc:302] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d12c7ef7fd4a497bb2d352f4f42c610c; no voters:
I20260926 16:35:50.667217 27819 leader_election.cc:288] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [CANDIDATE]: Term 3 election: Requested vote from peers
I20260926 16:35:50.667397 27823 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [term 3 FOLLOWER]: Leader election won for term 3
I20260926 16:35:50.667771 27823 raft_consensus.cc:686] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [term 3 LEADER]: Becoming Leader. State: Replica: d12c7ef7fd4a497bb2d352f4f42c610c, State: Running, Role: LEADER
I20260926 16:35:50.668183 27823 consensus_queue.cc:237] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d12c7ef7fd4a497bb2d352f4f42c610c" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37989 } }
I20260926 16:35:50.668426 27819 sys_catalog.cc:564] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:35:50.669529 27824 sys_catalog.cc:455] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "d12c7ef7fd4a497bb2d352f4f42c610c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d12c7ef7fd4a497bb2d352f4f42c610c" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37989 } } }
I20260926 16:35:50.669641 27824 sys_catalog.cc:458] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [sys.catalog]: This master's current role is: LEADER
I20260926 16:35:50.670481 27825 sys_catalog.cc:455] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d12c7ef7fd4a497bb2d352f4f42c610c. Latest consensus state: current_term: 3 leader_uuid: "d12c7ef7fd4a497bb2d352f4f42c610c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d12c7ef7fd4a497bb2d352f4f42c610c" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37989 } } }
I20260926 16:35:50.670810 27825 sys_catalog.cc:458] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c [sys.catalog]: This master's current role is: LEADER
I20260926 16:35:50.672870 27839 catalog_manager.cc:1259] Loaded cluster ID: 4bccdb7bda82407f820ba4ef70913ba4
I20260926 16:35:50.672945 27839 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c: loading cluster ID for follower catalog manager: success
I20260926 16:35:50.674728 27839 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d12c7ef7fd4a497bb2d352f4f42c610c: acquiring CA information for follower catalog manager: success
F20260926 16:35:50.675633 27839 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f2aa65a29ad google::LogMessage::Fail() at ??:0
@ 0x7f2aa65a588a google::LogMessage::SendToLog() at ??:0
@ 0x7f2aa65a24de google::LogMessage::Flush() at ??:0
@ 0x7f2aa65a3a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f2aa926618d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f2ab0f07989 kudu::master::CatalogManager::LoadTspkEntries() at ??:0
@ 0x7f2ab0ee4023 kudu::master::CatalogManager::PrepareFollowerTokenVerifier() at ??:0
@ 0x7f2ab0ee454b kudu::master::CatalogManager::PrepareFollower() at ??:0
@ 0x7f2ab0edd652 kudu::master::CatalogManagerBgTasks::Run() at ??:0
@ 0x7f2ab0edc9e9 _ZZN4kudu6master21CatalogManagerBgTasks4InitEvENKUlvE_clEv at ??:0
@ 0x7f2ab0f23541 _ZNSt17_Function_handlerIFvvEZN4kudu6master21CatalogManagerBgTasks4InitEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f2aa87e5512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f2aa9e296db start_thread at ??:0
@ 0x7f2aa50a061f clone at ??:0
W20260926 16:35:50.755993 27821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:35:50.756269 27821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:35:50.756341 27821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:35:50.760401 27821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:35:50.760481 27821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:35:50.760565 27821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.193
I20260926 16:35:50.765799 27821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:38229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.193:46177
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.193
--webserver_port=38551
--tserver_master_addrs=127.11.8.254:37989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27821
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.193
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:35:50.766901 27821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:35:50.767903 27821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:35:50.777074 27860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:35:50.777048 27862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:35:50.777068 27859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:35:50.777277 27821 server_base.cc:1034] running on GCE node
I20260926 16:35:50.777765 27821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:35:50.778465 27821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:35:50.779649 27821 hybrid_clock.cc:648] HybridClock initialized: now 1790440550779624 us; error 42 us; skew 500 ppm
Sep 26 16:35:50 dist-test-slave-2zvq krb5kdc[26675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440550, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:35:50.782449 27821 init.cc:377] Logged in from keytab as kudu/127.11.8.193@KRBTEST.COM (short username kudu)
I20260926 16:35:50.783293 27821 webserver.cc:460] Webserver started at http://127.11.8.193:38551/ using document root <none> and password file <none>
I20260926 16:35:50.783830 27821 fs_manager.cc:359] Metadata directory not provided
I20260926 16:35:50.783890 27821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:35:50.787653 27821 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:35:50.790570 27869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:35:50.792059 27821 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:35:50.792167 27821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/wal
uuid: "082a50743594445aacfab4de6c7d914f"
format_stamp: "Formatted at 2026-09-26 16:35:08 on dist-test-slave-2zvq"
server_key: "04c87e70745a85f80eede664d0e6f031"
server_key_iv: "cba335d4e28f3cc8d729c4887234d341"
server_key_version: "encryptionkey@0"
I20260926 16:35:50.792826 27821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:35:50.805208 27821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:35:50.808458 27821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:35:50.808651 27821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:35:50.809207 27821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:35:50.810295 27876 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260926 16:35:50.815644 27821 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 16:35:50.815743 27821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260926 16:35:50.815833 27821 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 16:35:50.817560 27821 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 16:35:50.817648 27821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260926 16:35:50.817916 27876 tablet_bootstrap.cc:492] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f: Bootstrap starting.
I20260926 16:35:50.828265 27821 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.193:46177
I20260926 16:35:50.828411 27983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.193:46177 every 8 connection(s)
I20260926 16:35:50.829317 27821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
I20260926 16:35:50.832311 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 27821
I20260926 16:35:50.832726 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.194:34339
--local_ip_for_outbound_sockets=127.11.8.194
--tserver_master_addrs=127.11.8.254:37989
--webserver_port=38161
--webserver_interface=127.11.8.194
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:38229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 16:35:50.839491 27876 log.cc:824] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f: Log is configured to *not* fsync() on all Append() calls
I20260926 16:35:50.848244 27876 tablet_bootstrap.cc:492] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 16:35:50.848628 27876 tablet_bootstrap.cc:492] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f: Bootstrap complete.
I20260926 16:35:50.850024 27876 ts_tablet_manager.cc:1397] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f: Time spent bootstrapping tablet: real 0.032s user 0.021s sys 0.007s
I20260926 16:35:50.853309 27876 raft_consensus.cc:348] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03003d7115c04b9d90c5c8d257aa5a33" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 38387 } } peers { permanent_uuid: "c85733551b2d4dd7b984240ad8ab3224" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34339 } } peers { permanent_uuid: "082a50743594445aacfab4de6c7d914f" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46177 } }
I20260926 16:35:50.853847 27876 raft_consensus.cc:729] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 082a50743594445aacfab4de6c7d914f, State: Initialized, Role: FOLLOWER
I20260926 16:35:50.854419 27876 consensus_queue.cc:260] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03003d7115c04b9d90c5c8d257aa5a33" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 38387 } } peers { permanent_uuid: "c85733551b2d4dd7b984240ad8ab3224" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34339 } } peers { permanent_uuid: "082a50743594445aacfab4de6c7d914f" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46177 } }
I20260926 16:35:50.855427 27876 ts_tablet_manager.cc:1428] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f: Time spent starting tablet: real 0.005s user 0.004s sys 0.004s
W20260926 16:35:50.892622 27986 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:35:50.831266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.193:53051)
0926 16:35:50.831746 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:50.831810 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0926 16:35:50.834102 (+ 2292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:35:50.891845 (+ 57743us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":143,"threads_started":1}
W20260926 16:35:50.893116 27984 heartbeater.cc:644] Failed to heartbeat to 127.11.8.254:37989 (0 consecutive failures): Network error: Failed to ping master at 127.11.8.254:37989: Client connection negotiation failed: client connection to 127.11.8.254:37989: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 16:35:50.893872 27986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:50.893645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.193:38995)
0926 16:35:50.893763 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:50.893822 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260926 16:35:50.894559 27986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:50.894378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.193:51061)
0926 16:35:50.894473 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:50.894515 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260926 16:35:50.894773 27984 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 16:35:50.947152 27987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:35:50.947407 27987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:35:50.947449 27987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:35:50.951457 27987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:35:50.951510 27987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:35:50.951567 27987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.194
I20260926 16:35:50.956688 27987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:38229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.194:34339
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.194
--webserver_port=38161
--tserver_master_addrs=127.11.8.254:37989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.194
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:35:50.957700 27987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:35:50.958760 27987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:35:50.968735 27995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:35:50.968721 27994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:35:50.968807 27987 server_base.cc:1034] running on GCE node
W20260926 16:35:50.968735 27997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:35:50.969401 27987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:35:50.969969 27987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:35:50.971127 27987 hybrid_clock.cc:648] HybridClock initialized: now 1790440550971114 us; error 25 us; skew 500 ppm
Sep 26 16:35:50 dist-test-slave-2zvq krb5kdc[26675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440550, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:35:50.973920 27987 init.cc:377] Logged in from keytab as kudu/127.11.8.194@KRBTEST.COM (short username kudu)
I20260926 16:35:50.974761 27987 webserver.cc:460] Webserver started at http://127.11.8.194:38161/ using document root <none> and password file <none>
I20260926 16:35:50.975276 27987 fs_manager.cc:359] Metadata directory not provided
I20260926 16:35:50.975337 27987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:35:50.979130 27987 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:35:50.981035 28004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:35:50.982218 27987 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:35:50.982400 27987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/wal
uuid: "c85733551b2d4dd7b984240ad8ab3224"
format_stamp: "Formatted at 2026-09-26 16:35:08 on dist-test-slave-2zvq"
server_key: "fb9e803b8efcad51e23c812fda8f78a8"
server_key_iv: "d3ebeccfddfb8d1249e0959af884e529"
server_key_version: "encryptionkey@0"
I20260926 16:35:50.982934 27987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:35:50.998953 27987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:35:51.002110 27987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:35:51.002336 27987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:35:51.002899 27987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:35:51.003669 27987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:35:51.003741 27987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:35:51.003818 27987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:35:51.003860 27987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:35:51.012557 27987 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.194:34339
I20260926 16:35:51.012583 28117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.194:34339 every 8 connection(s)
I20260926 16:35:51.013497 27987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
I20260926 16:35:51.016829 28120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:51.015458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.194:46425)
0926 16:35:51.016088 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:51.016187 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":109,"threads_started":1}
W20260926 16:35:51.017336 28118 heartbeater.cc:644] Failed to heartbeat to 127.11.8.254:37989 (0 consecutive failures): Network error: Failed to ping master at 127.11.8.254:37989: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
I20260926 16:35:51.017899 28120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:51.017694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.194:49947)
0926 16:35:51.017789 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:51.017834 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260926 16:35:51.018476 28120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:51.018309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.194:33537)
0926 16:35:51.018395 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:51.018433 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260926 16:35:51.018644 28118 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 16:35:51.019486 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 27987
I20260926 16:35:51.019852 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.195:38387
--local_ip_for_outbound_sockets=127.11.8.195
--tserver_master_addrs=127.11.8.254:37989
--webserver_port=38133
--webserver_interface=127.11.8.195
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:38229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:35:51.140961 28121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:35:51.141215 28121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:35:51.141258 28121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:35:51.145354 28121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:35:51.145411 28121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:35:51.145468 28121 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.195
I20260926 16:35:51.150640 28121 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:38229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.195:38387
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.195
--webserver_port=38133
--tserver_master_addrs=127.11.8.254:37989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.195
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:35:51.151602 28121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:35:51.152552 28121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:35:51.162340 28126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:35:51.162369 28129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:35:51.162374 28127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:35:51.162631 28121 server_base.cc:1034] running on GCE node
I20260926 16:35:51.163158 28121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:35:51.163852 28121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:35:51.165036 28121 hybrid_clock.cc:648] HybridClock initialized: now 1790440551165020 us; error 25 us; skew 500 ppm
Sep 26 16:35:51 dist-test-slave-2zvq krb5kdc[26675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440551, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:35:51.167889 28121 init.cc:377] Logged in from keytab as kudu/127.11.8.195@KRBTEST.COM (short username kudu)
I20260926 16:35:51.168680 28121 webserver.cc:460] Webserver started at http://127.11.8.195:38133/ using document root <none> and password file <none>
I20260926 16:35:51.169188 28121 fs_manager.cc:359] Metadata directory not provided
I20260926 16:35:51.169245 28121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:35:51.172844 28121 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:35:51.174821 28136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:35:51.176034 28121 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:35:51.176162 28121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/wal
uuid: "03003d7115c04b9d90c5c8d257aa5a33"
format_stamp: "Formatted at 2026-09-26 16:35:08 on dist-test-slave-2zvq"
server_key: "0a5993d3a7529ac1925c75221237d151"
server_key_iv: "3c2d75a4e0bb2641545b2e541fd7353b"
server_key_version: "encryptionkey@0"
I20260926 16:35:51.176678 28121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:35:51.213195 28121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:35:51.216555 28121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:35:51.216761 28121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:35:51.217356 28121 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:35:51.218168 28121 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:35:51.218236 28121 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:35:51.218334 28121 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:35:51.218379 28121 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:35:51.226663 28121 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.195:38387
I20260926 16:35:51.226689 28249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.195:38387 every 8 connection(s)
I20260926 16:35:51.227591 28121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
I20260926 16:35:51.231031 28252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:51.229586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.195:40273)
0926 16:35:51.230206 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:51.230367 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":107,"threads_started":1}
W20260926 16:35:51.231432 28250 heartbeater.cc:644] Failed to heartbeat to 127.11.8.254:37989 (0 consecutive failures): Network error: Failed to ping master at 127.11.8.254:37989: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
I20260926 16:35:51.231993 28252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:51.231785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.195:51101)
0926 16:35:51.231885 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:51.231936 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260926 16:35:51.232610 28252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:51.232418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.195:36513)
0926 16:35:51.232516 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:51.232558 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260926 16:35:51.232771 28250 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 16:35:51.236359 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 28121
I20260926 16:35:51.895707 28254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:51.895278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.193:58707)
0926 16:35:51.895551 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:51.895601 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":79,"threads_started":1}
I20260926 16:35:52.019619 28255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:52.019214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.194:58525)
0926 16:35:52.019470 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:52.019524 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":71,"threads_started":1}
I20260926 16:35:52.116173 28256 raft_consensus.cc:482] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:35:52.116504 28256 raft_consensus.cc:504] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03003d7115c04b9d90c5c8d257aa5a33" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 38387 } } peers { permanent_uuid: "c85733551b2d4dd7b984240ad8ab3224" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34339 } } peers { permanent_uuid: "082a50743594445aacfab4de6c7d914f" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46177 } }
I20260926 16:35:52.117784 28256 leader_election.cc:288] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 03003d7115c04b9d90c5c8d257aa5a33 (127.11.8.195:38387), c85733551b2d4dd7b984240ad8ab3224 (127.11.8.194:34339)
Sep 26 16:35:52 dist-test-slave-2zvq krb5kdc[26675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440550, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for kudu/127.11.8.194@KRBTEST.COM
Sep 26 16:35:52 dist-test-slave-2zvq krb5kdc[26675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440550, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for kudu/127.11.8.195@KRBTEST.COM
I20260926 16:35:52.131196 28254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:52.117946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:38387 (local address 127.11.8.193:51461)
0926 16:35:52.118114 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:52.118130 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:35:52.118234 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:35:52.119978 (+ 1744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:35:52.119984 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:35:52.120520 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:35:52.121005 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:35:52.121019 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:35:52.122303 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:35:52.122308 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:35:52.123220 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:35:52.123229 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:35:52.123905 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:35:52.123908 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:35:52.124109 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:52.124823 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:35:52.124838 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:35:52.126771 (+ 1933us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:35:52.129174 (+ 2403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:35:52.129177 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:35:52.129179 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:35:52.129433 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:35:52.129795 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:35:52.129797 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:35:52.129799 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:35:52.129864 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:35:52.130567 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:35:52.130569 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:35:52.130661 (+ 92us) client_negotiation.cc:769] Sending connection context
0926 16:35:52.130747 (+ 86us) client_negotiation.cc:240] Negotiation successful
0926 16:35:52.130986 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":153,"spinlock_wait_cycles":7424}
I20260926 16:35:52.131281 28259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:52.118387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:41529 (local address 127.11.8.194:34339)
0926 16:35:52.118714 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0926 16:35:52.118732 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 16:35:52.118762 (+ 30us) server_negotiation.cc:407] Connection header received
0926 16:35:52.119407 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:35:52.119413 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:35:52.119698 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:35:52.119990 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:35:52.121189 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:52.122112 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:52.123409 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:52.123845 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:52.124021 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:52.126649 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:35:52.126656 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:35:52.126672 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:35:52.126708 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:35:52.128400 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:35:52.129010 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:35:52.129017 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:35:52.129023 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:35:52.129176 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:35:52.129555 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:35:52.129558 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:35:52.129560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:35:52.130077 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:35:52.130208 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0926 16:35:52.130812 (+ 604us) server_negotiation.cc:299] Negotiation successful
0926 16:35:52.131101 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":96,"threads_started":1}
I20260926 16:35:52.131281 28258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:52.118094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:51461 (local address 127.11.8.195:38387)
0926 16:35:52.118503 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0926 16:35:52.118520 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 16:35:52.118568 (+ 48us) server_negotiation.cc:407] Connection header received
0926 16:35:52.119275 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:35:52.119281 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:35:52.119631 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:35:52.119946 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:35:52.121173 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:52.122114 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:52.123337 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:35:52.123783 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:35:52.123962 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:52.126956 (+ 2994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:35:52.126960 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:35:52.126970 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:35:52.126996 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:35:52.129066 (+ 2070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:35:52.129566 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:35:52.129571 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:35:52.129576 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:35:52.129701 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:35:52.129977 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:35:52.129981 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:35:52.129982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:35:52.130469 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:35:52.130569 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 16:35:52.130900 (+ 331us) server_negotiation.cc:299] Negotiation successful
0926 16:35:52.131115 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":94,"threads_started":1}
I20260926 16:35:52.131196 28257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:52.118014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:34339 (local address 127.11.8.193:41529)
0926 16:35:52.118504 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:52.118517 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:35:52.118607 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:35:52.119979 (+ 1372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:35:52.119985 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:35:52.120521 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:35:52.121005 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:35:52.121020 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:35:52.122261 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:35:52.122314 (+ 53us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:35:52.123228 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:35:52.123238 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:35:52.123975 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:35:52.123978 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:35:52.124145 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:35:52.124881 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:35:52.124892 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:35:52.126407 (+ 1515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:35:52.128518 (+ 2111us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:35:52.128524 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:35:52.128537 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:35:52.128868 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:35:52.129269 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:35:52.129275 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:35:52.129278 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:35:52.129441 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:35:52.130184 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:35:52.130189 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:35:52.130462 (+ 273us) client_negotiation.cc:769] Sending connection context
0926 16:35:52.130642 (+ 180us) client_negotiation.cc:240] Negotiation successful
0926 16:35:52.131038 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"mutex_wait_us":342,"spinlock_wait_cycles":20864,"thread_start_us":94,"threads_started":1}
W20260926 16:35:52.134105 27870 leader_election.cc:341] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c85733551b2d4dd7b984240ad8ab3224 (127.11.8.194:34339): Not found: Tablet not found: 855a4d10b4e342e9b4eefdae4c493934
W20260926 16:35:52.134828 27871 leader_election.cc:341] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 03003d7115c04b9d90c5c8d257aa5a33 (127.11.8.195:38387): Not found: Tablet not found: 855a4d10b4e342e9b4eefdae4c493934
I20260926 16:35:52.134910 27871 leader_election.cc:302] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 082a50743594445aacfab4de6c7d914f; no voters: 03003d7115c04b9d90c5c8d257aa5a33, c85733551b2d4dd7b984240ad8ab3224
I20260926 16:35:52.135138 28256 raft_consensus.cc:2726] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 16:35:52.233690 28260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:52.233246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.195:34613)
0926 16:35:52.233542 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:52.233595 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":107,"threads_started":1}
I20260926 16:35:52.896835 28261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:52.896426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.193:34203)
0926 16:35:52.896669 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:52.896727 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":73,"threads_started":1}
I20260926 16:35:53.020792 28262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:53.020405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.194:42307)
0926 16:35:53.020635 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:53.020706 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":71,"threads_started":1}
I20260926 16:35:53.234794 28263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:53.234421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.195:37327)
0926 16:35:53.234642 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:53.234695 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":68,"threads_started":1}
I20260926 16:35:53.897964 28264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:53.897570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.193:44773)
0926 16:35:53.897809 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:53.897871 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":76,"threads_started":1}
I20260926 16:35:53.981500 28265 raft_consensus.cc:482] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:35:53.981644 28265 raft_consensus.cc:504] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03003d7115c04b9d90c5c8d257aa5a33" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 38387 } } peers { permanent_uuid: "c85733551b2d4dd7b984240ad8ab3224" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34339 } } peers { permanent_uuid: "082a50743594445aacfab4de6c7d914f" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46177 } }
I20260926 16:35:53.982059 28265 leader_election.cc:288] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 03003d7115c04b9d90c5c8d257aa5a33 (127.11.8.195:38387), c85733551b2d4dd7b984240ad8ab3224 (127.11.8.194:34339)
W20260926 16:35:53.982935 27871 leader_election.cc:341] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 03003d7115c04b9d90c5c8d257aa5a33 (127.11.8.195:38387): Not found: Tablet not found: 855a4d10b4e342e9b4eefdae4c493934
W20260926 16:35:53.983019 27870 leader_election.cc:341] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c85733551b2d4dd7b984240ad8ab3224 (127.11.8.194:34339): Not found: Tablet not found: 855a4d10b4e342e9b4eefdae4c493934
I20260926 16:35:53.983121 27870 leader_election.cc:302] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 082a50743594445aacfab4de6c7d914f; no voters: 03003d7115c04b9d90c5c8d257aa5a33, c85733551b2d4dd7b984240ad8ab3224
I20260926 16:35:53.983299 28265 raft_consensus.cc:2726] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 16:35:54.022002 28266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:54.021616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.194:44125)
0926 16:35:54.021853 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:54.021905 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":82,"threads_started":1}
I20260926 16:35:54.235912 28267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:54.235559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.195:36105)
0926 16:35:54.235785 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:54.235832 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":84,"threads_started":1}
I20260926 16:35:54.899041 28268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:54.898632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.193:45333)
0926 16:35:54.898892 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:54.898947 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":71,"threads_started":1}
I20260926 16:35:55.023190 28269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:55.022792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.194:36481)
0926 16:35:55.023053 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:55.023104 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":102,"threads_started":1}
I20260926 16:35:55.237088 28270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:55.236593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.195:45747)
0926 16:35:55.236941 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:55.236998 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":207,"threads_started":1}
I20260926 16:35:55.900405 28271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:55.899942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.193:41507)
0926 16:35:55.900240 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:55.900299 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":86,"threads_started":1}
I20260926 16:35:56.024330 28272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:56.023939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.194:50265)
0926 16:35:56.024191 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:56.024244 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":86,"threads_started":1}
I20260926 16:35:56.238238 28273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:56.237821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.195:58607)
0926 16:35:56.238058 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:56.238111 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":77,"threads_started":1}
I20260926 16:35:56.855425 28274 raft_consensus.cc:482] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:35:56.855549 28274 raft_consensus.cc:504] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03003d7115c04b9d90c5c8d257aa5a33" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 38387 } } peers { permanent_uuid: "c85733551b2d4dd7b984240ad8ab3224" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34339 } } peers { permanent_uuid: "082a50743594445aacfab4de6c7d914f" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46177 } }
I20260926 16:35:56.855988 28274 leader_election.cc:288] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 03003d7115c04b9d90c5c8d257aa5a33 (127.11.8.195:38387), c85733551b2d4dd7b984240ad8ab3224 (127.11.8.194:34339)
W20260926 16:35:56.856724 27870 leader_election.cc:341] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c85733551b2d4dd7b984240ad8ab3224 (127.11.8.194:34339): Not found: Tablet not found: 855a4d10b4e342e9b4eefdae4c493934
W20260926 16:35:56.856819 27871 leader_election.cc:341] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 03003d7115c04b9d90c5c8d257aa5a33 (127.11.8.195:38387): Not found: Tablet not found: 855a4d10b4e342e9b4eefdae4c493934
I20260926 16:35:56.856933 27871 leader_election.cc:302] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 082a50743594445aacfab4de6c7d914f; no voters: 03003d7115c04b9d90c5c8d257aa5a33, c85733551b2d4dd7b984240ad8ab3224
I20260926 16:35:56.857110 28274 raft_consensus.cc:2726] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 16:35:56.901502 28275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:56.901106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.193:52171)
0926 16:35:56.901352 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:56.901408 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":69,"threads_started":1}
I20260926 16:35:57.025455 28276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:57.025033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.194:38411)
0926 16:35:57.025310 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:57.025365 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":87,"threads_started":1}
I20260926 16:35:57.239403 28277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:57.238981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.195:46403)
0926 16:35:57.239246 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:57.239300 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":76,"threads_started":1}
I20260926 16:35:57.902709 28278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:57.902206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.193:36381)
0926 16:35:57.902558 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:57.902612 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":188,"threads_started":1}
I20260926 16:35:58.026625 28279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:58.026172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.194:59663)
0926 16:35:58.026484 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:58.026537 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":115,"threads_started":1}
I20260926 16:35:58.240517 28280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:58.240082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.195:33083)
0926 16:35:58.240374 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:58.240427 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":70,"threads_started":1}
I20260926 16:35:58.863513 28281 raft_consensus.cc:482] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:35:58.863633 28281 raft_consensus.cc:504] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03003d7115c04b9d90c5c8d257aa5a33" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 38387 } } peers { permanent_uuid: "c85733551b2d4dd7b984240ad8ab3224" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34339 } } peers { permanent_uuid: "082a50743594445aacfab4de6c7d914f" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46177 } }
I20260926 16:35:58.864097 28281 leader_election.cc:288] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 03003d7115c04b9d90c5c8d257aa5a33 (127.11.8.195:38387), c85733551b2d4dd7b984240ad8ab3224 (127.11.8.194:34339)
W20260926 16:35:58.864917 27871 leader_election.cc:341] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 03003d7115c04b9d90c5c8d257aa5a33 (127.11.8.195:38387): Not found: Tablet not found: 855a4d10b4e342e9b4eefdae4c493934
W20260926 16:35:58.865024 27870 leader_election.cc:341] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c85733551b2d4dd7b984240ad8ab3224 (127.11.8.194:34339): Not found: Tablet not found: 855a4d10b4e342e9b4eefdae4c493934
I20260926 16:35:58.865091 27870 leader_election.cc:302] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 082a50743594445aacfab4de6c7d914f; no voters: 03003d7115c04b9d90c5c8d257aa5a33, c85733551b2d4dd7b984240ad8ab3224
I20260926 16:35:58.865214 28281 raft_consensus.cc:2726] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 16:35:58.903724 28282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:58.903357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.193:53315)
0926 16:35:58.903583 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:58.903632 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":86,"threads_started":1}
I20260926 16:35:59.027833 28283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:59.027417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.194:59433)
0926 16:35:59.027664 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:59.027719 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":75,"threads_started":1}
I20260926 16:35:59.241745 28284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:59.241282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.195:41049)
0926 16:35:59.241544 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:59.241596 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":92,"threads_started":1}
I20260926 16:35:59.904796 28285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:35:59.904409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.193:45017)
0926 16:35:59.904654 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0926 16:35:59.904703 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":92,"threads_started":1}
I20260926 16:36:00.028976 28286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:00.028571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.194:48551)
0926 16:36:00.028826 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:00.028885 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":76,"threads_started":1}
I20260926 16:36:00.242815 28287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:00.242453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.195:35631)
0926 16:36:00.242679 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:00.242731 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":71,"threads_started":1}
I20260926 16:36:00.420898 28288 raft_consensus.cc:482] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:36:00.421036 28288 raft_consensus.cc:504] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03003d7115c04b9d90c5c8d257aa5a33" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 38387 } } peers { permanent_uuid: "c85733551b2d4dd7b984240ad8ab3224" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34339 } } peers { permanent_uuid: "082a50743594445aacfab4de6c7d914f" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46177 } }
I20260926 16:36:00.421438 28288 leader_election.cc:288] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 03003d7115c04b9d90c5c8d257aa5a33 (127.11.8.195:38387), c85733551b2d4dd7b984240ad8ab3224 (127.11.8.194:34339)
W20260926 16:36:00.422253 27870 leader_election.cc:341] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c85733551b2d4dd7b984240ad8ab3224 (127.11.8.194:34339): Not found: Tablet not found: 855a4d10b4e342e9b4eefdae4c493934
W20260926 16:36:00.422463 27871 leader_election.cc:341] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 03003d7115c04b9d90c5c8d257aa5a33 (127.11.8.195:38387): Not found: Tablet not found: 855a4d10b4e342e9b4eefdae4c493934
I20260926 16:36:00.422554 27871 leader_election.cc:302] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 082a50743594445aacfab4de6c7d914f; no voters: 03003d7115c04b9d90c5c8d257aa5a33, c85733551b2d4dd7b984240ad8ab3224
I20260926 16:36:00.422712 28288 raft_consensus.cc:2726] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 16:36:00.906100 28289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:00.905691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.193:42655)
0926 16:36:00.905946 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:00.906003 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":84,"threads_started":1}
I20260926 16:36:01.030148 28290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:01.029765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.194:55911)
0926 16:36:01.030007 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:01.030058 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":72,"threads_started":1}
I20260926 16:36:01.243916 28291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:01.243555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.195:41675)
0926 16:36:01.243776 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:01.243826 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":76,"threads_started":1}
I20260926 16:36:01.907404 28292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:01.906959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.193:55065)
0926 16:36:01.907226 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:01.907305 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":72,"threads_started":1}
I20260926 16:36:02.031522 28293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:02.031048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.194:36267)
0926 16:36:02.031343 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:02.031424 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":76,"threads_started":1}
I20260926 16:36:02.245136 28294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:02.244713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.195:38605)
0926 16:36:02.244977 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:02.245038 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":90,"threads_started":1}
I20260926 16:36:02.908715 28295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:02.908267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.193:52663)
0926 16:36:02.908555 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:02.908618 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":94,"threads_started":1}
I20260926 16:36:03.032745 28296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:03.032351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.194:55663)
0926 16:36:03.032596 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:03.032651 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":82,"threads_started":1}
I20260926 16:36:03.212087 28297 raft_consensus.cc:482] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:36:03.212218 28297 raft_consensus.cc:504] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03003d7115c04b9d90c5c8d257aa5a33" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 38387 } } peers { permanent_uuid: "c85733551b2d4dd7b984240ad8ab3224" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 34339 } } peers { permanent_uuid: "082a50743594445aacfab4de6c7d914f" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46177 } }
I20260926 16:36:03.212666 28297 leader_election.cc:288] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 03003d7115c04b9d90c5c8d257aa5a33 (127.11.8.195:38387), c85733551b2d4dd7b984240ad8ab3224 (127.11.8.194:34339)
W20260926 16:36:03.213493 27870 leader_election.cc:341] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c85733551b2d4dd7b984240ad8ab3224 (127.11.8.194:34339): Not found: Tablet not found: 855a4d10b4e342e9b4eefdae4c493934
W20260926 16:36:03.213594 27871 leader_election.cc:341] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 03003d7115c04b9d90c5c8d257aa5a33 (127.11.8.195:38387): Not found: Tablet not found: 855a4d10b4e342e9b4eefdae4c493934
I20260926 16:36:03.213660 27871 leader_election.cc:302] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 082a50743594445aacfab4de6c7d914f; no voters: 03003d7115c04b9d90c5c8d257aa5a33, c85733551b2d4dd7b984240ad8ab3224
I20260926 16:36:03.213799 28297 raft_consensus.cc:2726] T 855a4d10b4e342e9b4eefdae4c493934 P 082a50743594445aacfab4de6c7d914f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 16:36:03.246222 28298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:03.245841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.195:39631)
0926 16:36:03.246088 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:03.246135 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":93,"threads_started":1}
I20260926 16:36:03.909860 28299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:03.909445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.193:41183)
0926 16:36:03.909708 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:03.909767 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":75,"threads_started":1}
I20260926 16:36:04.033828 28300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:04.033443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.194:45561)
0926 16:36:04.033684 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:04.033735 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":87,"threads_started":1}
I20260926 16:36:04.247494 28301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:04.247004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.195:36173)
0926 16:36:04.247251 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:04.247370 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":71,"threads_started":1}
I20260926 16:36:04.911145 28302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:04.910716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.193:34795)
0926 16:36:04.910978 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:04.911039 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":75,"threads_started":1}
I20260926 16:36:05.035065 28303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:05.034651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.194:32941)
0926 16:36:05.034913 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:05.034968 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":75,"threads_started":1}
I20260926 16:36:05.248636 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:05.248247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.195:45177)
0926 16:36:05.248491 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:05.248544 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":72,"threads_started":1}
I20260926 16:36:05.912606 28305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:05.912168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.193:60491)
0926 16:36:05.912461 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:05.912514 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":104,"threads_started":1}
I20260926 16:36:06.036435 28306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:06.036001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.194:41469)
0926 16:36:06.036260 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:06.036313 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":75,"threads_started":1}
I20260926 16:36:06.244213 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 27821
I20260926 16:36:06.250087 28307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:06.249576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37989 (local address 127.11.8.195:50665)
0926 16:36:06.249868 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:06.249946 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.254:37989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":69,"threads_started":1}
I20260926 16:36:06.250626 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 27987
I20260926 16:36:06.255790 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 28121
2026-09-26T16:36:06Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59319 ms)
[----------] 30 tests from SecurityITest (241427 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:36:06 dist-test-slave-2zvq krb5kdc[28310](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:36:06 dist-test-slave-2zvq krb5kdc[28310](info): set up 2 sockets
Sep 26 16:36:06 dist-test-slave-2zvq krb5kdc[28310](info): commencing operation
krb5kdc: starting...
W20260926 16:36:07.307669 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:36:07 dist-test-slave-2zvq krb5kdc[28310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:36:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:36:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
I20260926 16:36:07.426801 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:39439
--webserver_interface=127.11.8.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:39349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:39439
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:36:07.536422 28323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:07.536716 28323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:07.536796 28323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:07.540808 28323 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:36:07.540887 28323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:07.540923 28323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:07.540961 28323 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:36:07.540992 28323 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:36:07.546128 28323 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:39349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:39439
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:39439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:07.547147 28323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:07.548141 28323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:07.556366 28331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:07.556455 28323 server_base.cc:1034] running on GCE node
W20260926 16:36:07.556473 28328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:07.556367 28329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:07.557201 28323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:07.558235 28323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:07.559438 28323 hybrid_clock.cc:648] HybridClock initialized: now 1790440567559425 us; error 25 us; skew 500 ppm
Sep 26 16:36:07 dist-test-slave-2zvq krb5kdc[28310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440567, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:36:07.561969 28323 init.cc:377] Logged in from keytab as kudu/127.11.8.254@KRBTEST.COM (short username kudu)
I20260926 16:36:07.562834 28323 webserver.cc:460] Webserver started at http://127.11.8.254:39483/ using document root <none> and password file <none>
I20260926 16:36:07.563311 28323 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:07.563365 28323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:07.563530 28323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:07.565414 28323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "999b5b77354046e6bbc58955d32b6c9e"
format_stamp: "Formatted at 2026-09-26 16:36:07 on dist-test-slave-2zvq"
server_key: "fe4972ae971153232afa053756ee527a"
server_key_iv: "cf83668b37d09c235871ef9ae304a1f0"
server_key_version: "encryptionkey@0"
I20260926 16:36:07.565894 28323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "999b5b77354046e6bbc58955d32b6c9e"
format_stamp: "Formatted at 2026-09-26 16:36:07 on dist-test-slave-2zvq"
server_key: "fe4972ae971153232afa053756ee527a"
server_key_iv: "cf83668b37d09c235871ef9ae304a1f0"
server_key_version: "encryptionkey@0"
I20260926 16:36:07.569298 28323 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260926 16:36:07.571564 28338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:07.572757 28323 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 16:36:07.572876 28323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "999b5b77354046e6bbc58955d32b6c9e"
format_stamp: "Formatted at 2026-09-26 16:36:07 on dist-test-slave-2zvq"
server_key: "fe4972ae971153232afa053756ee527a"
server_key_iv: "cf83668b37d09c235871ef9ae304a1f0"
server_key_version: "encryptionkey@0"
I20260926 16:36:07.572978 28323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:07.589416 28323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:07.592724 28323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:07.592911 28323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:07.600214 28323 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:39439
I20260926 16:36:07.600234 28390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:39439 every 8 connection(s)
I20260926 16:36:07.601176 28323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:36:07.602937 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 28323
I20260926 16:36:07.603056 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:36:07.603361 11299 external_mini_cluster.cc:1428] Setting key d4635884bd3b790900d02f1d7cc47850
I20260926 16:36:07.604254 28391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:07.610912 28391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 999b5b77354046e6bbc58955d32b6c9e: Bootstrap starting.
Sep 26 16:36:07 dist-test-slave-2zvq krb5kdc[28310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:36:07.613157 28391 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 999b5b77354046e6bbc58955d32b6c9e: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:07.614071 28391 log.cc:824] T 00000000000000000000000000000000 P 999b5b77354046e6bbc58955d32b6c9e: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:07.616360 28391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 999b5b77354046e6bbc58955d32b6c9e: No bootstrap required, opened a new log
I20260926 16:36:07.618360 28394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:07.604787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35862 (local address 127.11.8.254:39439)
0926 16:36:07.605204 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:07.605224 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:07.605266 (+ 42us) server_negotiation.cc:407] Connection header received
0926 16:36:07.606139 (+ 873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:07.606161 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:07.606590 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:07.606964 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:07.607632 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:07.609103 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:07.610020 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:07.610706 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:07.610881 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:07.613296 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:07.613304 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:07.613320 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:07.613352 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:07.615328 (+ 1976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:07.615815 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:07.615820 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:07.615825 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:07.615958 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:07.616213 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:07.616216 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:07.616217 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:07.616708 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:07.616890 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:07.617130 (+ 240us) server_negotiation.cc:299] Negotiation successful
0926 16:36:07.617416 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":118,"threads_started":1}
I20260926 16:36:07.620692 28391 raft_consensus.cc:348] T 00000000000000000000000000000000 P 999b5b77354046e6bbc58955d32b6c9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "999b5b77354046e6bbc58955d32b6c9e" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 39439 } }
I20260926 16:36:07.620900 28391 raft_consensus.cc:374] T 00000000000000000000000000000000 P 999b5b77354046e6bbc58955d32b6c9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:36:07.620940 28391 raft_consensus.cc:729] T 00000000000000000000000000000000 P 999b5b77354046e6bbc58955d32b6c9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 999b5b77354046e6bbc58955d32b6c9e, State: Initialized, Role: FOLLOWER
I20260926 16:36:07.621340 28391 consensus_queue.cc:260] T 00000000000000000000000000000000 P 999b5b77354046e6bbc58955d32b6c9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "999b5b77354046e6bbc58955d32b6c9e" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 39439 } }
I20260926 16:36:07.621456 28391 raft_consensus.cc:388] T 00000000000000000000000000000000 P 999b5b77354046e6bbc58955d32b6c9e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:36:07.621500 28391 raft_consensus.cc:482] T 00000000000000000000000000000000 P 999b5b77354046e6bbc58955d32b6c9e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:36:07.621567 28391 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 999b5b77354046e6bbc58955d32b6c9e [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:07.622407 28391 raft_consensus.cc:504] T 00000000000000000000000000000000 P 999b5b77354046e6bbc58955d32b6c9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "999b5b77354046e6bbc58955d32b6c9e" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 39439 } }
I20260926 16:36:07.622689 28391 leader_election.cc:302] T 00000000000000000000000000000000 P 999b5b77354046e6bbc58955d32b6c9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 999b5b77354046e6bbc58955d32b6c9e; no voters:
I20260926 16:36:07.622936 28391 leader_election.cc:288] T 00000000000000000000000000000000 P 999b5b77354046e6bbc58955d32b6c9e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:36:07.623076 28396 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 999b5b77354046e6bbc58955d32b6c9e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:36:07.623333 28396 raft_consensus.cc:686] T 00000000000000000000000000000000 P 999b5b77354046e6bbc58955d32b6c9e [term 1 LEADER]: Becoming Leader. State: Replica: 999b5b77354046e6bbc58955d32b6c9e, State: Running, Role: LEADER
I20260926 16:36:07.623663 28396 consensus_queue.cc:237] T 00000000000000000000000000000000 P 999b5b77354046e6bbc58955d32b6c9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "999b5b77354046e6bbc58955d32b6c9e" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 39439 } }
I20260926 16:36:07.624145 28391 sys_catalog.cc:564] T 00000000000000000000000000000000 P 999b5b77354046e6bbc58955d32b6c9e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:36:07.625449 28397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 999b5b77354046e6bbc58955d32b6c9e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "999b5b77354046e6bbc58955d32b6c9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "999b5b77354046e6bbc58955d32b6c9e" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 39439 } } }
I20260926 16:36:07.625568 28397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 999b5b77354046e6bbc58955d32b6c9e [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:07.625931 28398 sys_catalog.cc:455] T 00000000000000000000000000000000 P 999b5b77354046e6bbc58955d32b6c9e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 999b5b77354046e6bbc58955d32b6c9e. Latest consensus state: current_term: 1 leader_uuid: "999b5b77354046e6bbc58955d32b6c9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "999b5b77354046e6bbc58955d32b6c9e" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 39439 } } }
I20260926 16:36:07.625977 28405 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:36:07.626016 28398 sys_catalog.cc:458] T 00000000000000000000000000000000 P 999b5b77354046e6bbc58955d32b6c9e [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:07.628522 28405 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:36:07.634034 28405 catalog_manager.cc:1347] Generated new cluster ID: 90c00d90ec8f49b7b801f13625478cea
I20260926 16:36:07.634110 28405 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:36:07.651394 28405 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:36:07.652392 28405 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:36:07.657881 28405 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 999b5b77354046e6bbc58955d32b6c9e: Generated new TSK 0
I20260926 16:36:07.658596 28405 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.8.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab.
Entry for principal kudu/127.11.8.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab.
I20260926 16:36:07.690701 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.193:0
--local_ip_for_outbound_sockets=127.11.8.193
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:39439
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:39349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:36:07.798059 28417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:07.798295 28417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:07.798337 28417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:07.802436 28417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:07.802485 28417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:07.802569 28417 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.193
I20260926 16:36:07.807615 28417 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:39349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:39439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28417
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.193
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:07.808537 28417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:07.809487 28417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:07.818701 28423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:07.818836 28417 server_base.cc:1034] running on GCE node
W20260926 16:36:07.818768 28425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:07.818701 28422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:07.819366 28417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:07.819947 28417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:07.821128 28417 hybrid_clock.cc:648] HybridClock initialized: now 1790440567821119 us; error 31 us; skew 500 ppm
Sep 26 16:36:07 dist-test-slave-2zvq krb5kdc[28310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440567, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:36:07.823889 28417 init.cc:377] Logged in from keytab as kudu/127.11.8.193@KRBTEST.COM (short username kudu)
I20260926 16:36:07.824860 28417 webserver.cc:460] Webserver started at http://127.11.8.193:38249/ using document root <none> and password file <none>
I20260926 16:36:07.825413 28417 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:07.825500 28417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:07.825735 28417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:07.827754 28417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-0/data/instance:
uuid: "3b4e066ad2d5493697f06b41e2e08066"
format_stamp: "Formatted at 2026-09-26 16:36:07 on dist-test-slave-2zvq"
server_key: "198cf69b8b37ae7d58579e7576d5fe12"
server_key_iv: "94f4ca70dfb1caab00c5f1ec6e238ccf"
server_key_version: "encryptionkey@0"
I20260926 16:36:07.828295 28417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance:
uuid: "3b4e066ad2d5493697f06b41e2e08066"
format_stamp: "Formatted at 2026-09-26 16:36:07 on dist-test-slave-2zvq"
server_key: "198cf69b8b37ae7d58579e7576d5fe12"
server_key_iv: "94f4ca70dfb1caab00c5f1ec6e238ccf"
server_key_version: "encryptionkey@0"
I20260926 16:36:07.831928 28417 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.001s
I20260926 16:36:07.834328 28432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:07.835469 28417 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:36:07.835592 28417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-0/wal
uuid: "3b4e066ad2d5493697f06b41e2e08066"
format_stamp: "Formatted at 2026-09-26 16:36:07 on dist-test-slave-2zvq"
server_key: "198cf69b8b37ae7d58579e7576d5fe12"
server_key_iv: "94f4ca70dfb1caab00c5f1ec6e238ccf"
server_key_version: "encryptionkey@0"
I20260926 16:36:07.835696 28417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:07.868510 28417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:07.871834 28417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:07.872035 28417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:07.872618 28417 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:36:07.873412 28417 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:36:07.873477 28417 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:07.873545 28417 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:36:07.873592 28417 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:07.882797 28417 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.193:42535
I20260926 16:36:07.882815 28545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.193:42535 every 8 connection(s)
I20260926 16:36:07.883651 28417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
I20260926 16:36:07.886631 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 28417
I20260926 16:36:07.886759 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance
I20260926 16:36:07.887045 11299 external_mini_cluster.cc:1428] Setting key 33a6dcb1a11d8457727db45f5cffd438
Sep 26 16:36:07 dist-test-slave-2zvq krb5kdc[28310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440567, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.8.194@KRBTEST.COM; defaulting to no policy
I20260926 16:36:07.897277 28394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:07.885399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:38817 (local address 127.11.8.254:39439)
0926 16:36:07.885536 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:07.885540 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:07.886528 (+ 988us) server_negotiation.cc:407] Connection header received
0926 16:36:07.887603 (+ 1075us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:07.887607 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:07.887659 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:07.887746 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:07.889110 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:07.889643 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:07.890859 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:07.891102 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:07.891213 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:07.893600 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:07.893604 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:07.893607 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:07.893631 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:07.895162 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:07.895687 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:07.895692 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:07.895694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:07.895805 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:07.896133 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:07.896138 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:07.896140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:07.896321 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:07.896427 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:07.896977 (+ 550us) server_negotiation.cc:299] Negotiation successful
0926 16:36:07.897096 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 16:36:07.898106 28548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:07.885691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:39439 (local address 127.11.8.193:38817)
0926 16:36:07.886364 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:07.886405 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:07.887374 (+ 969us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:07.887982 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:07.887991 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:07.888468 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:36:07.888955 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:07.888969 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:07.889777 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:07.889781 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:07.890719 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:07.890728 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:07.891248 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:07.891252 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:07.891421 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:07.892047 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:36:07.892062 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:36:07.893445 (+ 1383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:36:07.895315 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:07.895321 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:07.895334 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:07.895584 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:07.895895 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:07.895899 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:07.895901 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:07.896037 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:07.896426 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:36:07.896434 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:36:07.896721 (+ 287us) client_negotiation.cc:769] Sending connection context
0926 16:36:07.896938 (+ 217us) client_negotiation.cc:240] Negotiation successful
0926 16:36:07.897233 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":103,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.194@KRBTEST.COM" created.
I20260926 16:36:07.899303 28546 heartbeater.cc:347] Connected to a master server at 127.11.8.254:39439
I20260926 16:36:07.899531 28546 heartbeater.cc:464] Registering TS with master...
I20260926 16:36:07.899996 28546 heartbeater.cc:511] Master 127.11.8.254:39439 requested a full tablet report, sending...
I20260926 16:36:07.901468 28355 ts_manager.cc:194] Registered new tserver with Master: 3b4e066ad2d5493697f06b41e2e08066 (127.11.8.193:42535)
I20260926 16:36:07.902657 28355 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.193@KRBTEST.COM'} at 127.11.8.193:38817
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab.
Entry for principal kudu/127.11.8.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab.
I20260926 16:36:07.913154 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.194:0
--local_ip_for_outbound_sockets=127.11.8.194
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:39439
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:39349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:36:08.021576 28551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:08.021801 28551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:08.021842 28551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:08.025918 28551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:08.025972 28551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:08.026028 28551 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.194
I20260926 16:36:08.031165 28551 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:39349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:39439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.194
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:08.032087 28551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:08.033021 28551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:08.042151 28556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:08.042164 28557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:08.042160 28559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:08.042507 28551 server_base.cc:1034] running on GCE node
I20260926 16:36:08.042938 28551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:08.043560 28551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:08.044775 28551 hybrid_clock.cc:648] HybridClock initialized: now 1790440568044766 us; error 31 us; skew 500 ppm
Sep 26 16:36:08 dist-test-slave-2zvq krb5kdc[28310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440568, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:36:08.047817 28551 init.cc:377] Logged in from keytab as kudu/127.11.8.194@KRBTEST.COM (short username kudu)
I20260926 16:36:08.048694 28551 webserver.cc:460] Webserver started at http://127.11.8.194:35057/ using document root <none> and password file <none>
I20260926 16:36:08.049268 28551 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:08.049360 28551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:08.049588 28551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:08.051656 28551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-1/data/instance:
uuid: "16f6725aa1824e67b3d8f0e5c1bb84b4"
format_stamp: "Formatted at 2026-09-26 16:36:08 on dist-test-slave-2zvq"
server_key: "07d06f62757d2827d1e8962f9b7b6485"
server_key_iv: "1fb4b784944af8a0ec63ce716c4037c8"
server_key_version: "encryptionkey@0"
I20260926 16:36:08.052235 28551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance:
uuid: "16f6725aa1824e67b3d8f0e5c1bb84b4"
format_stamp: "Formatted at 2026-09-26 16:36:08 on dist-test-slave-2zvq"
server_key: "07d06f62757d2827d1e8962f9b7b6485"
server_key_iv: "1fb4b784944af8a0ec63ce716c4037c8"
server_key_version: "encryptionkey@0"
I20260926 16:36:08.055939 28551 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 16:36:08.058410 28566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:08.059773 28551 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260926 16:36:08.059933 28551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-1/wal
uuid: "16f6725aa1824e67b3d8f0e5c1bb84b4"
format_stamp: "Formatted at 2026-09-26 16:36:08 on dist-test-slave-2zvq"
server_key: "07d06f62757d2827d1e8962f9b7b6485"
server_key_iv: "1fb4b784944af8a0ec63ce716c4037c8"
server_key_version: "encryptionkey@0"
I20260926 16:36:08.060057 28551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:08.078379 28551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:08.081359 28551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:08.081568 28551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:08.082172 28551 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:36:08.083053 28551 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:36:08.083135 28551 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:08.083211 28551 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:36:08.083268 28551 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:08.093143 28551 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.194:38085
I20260926 16:36:08.093233 28679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.194:38085 every 8 connection(s)
I20260926 16:36:08.094584 28551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
I20260926 16:36:08.099202 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 28551
I20260926 16:36:08.099347 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance
I20260926 16:36:08.099702 11299 external_mini_cluster.cc:1428] Setting key 2dfa45485f57020dfbc2bc05b1514eaf
Sep 26 16:36:08 dist-test-slave-2zvq krb5kdc[28310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440568, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.8.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.195@KRBTEST.COM" created.
I20260926 16:36:08.113078 28394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:08.096958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:35783 (local address 127.11.8.254:39439)
0926 16:36:08.097164 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:08.097169 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:08.099003 (+ 1834us) server_negotiation.cc:407] Connection header received
0926 16:36:08.100387 (+ 1384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:08.100395 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:08.100489 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:08.100634 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:08.103475 (+ 2841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:08.104055 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:08.106068 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:08.106348 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:08.106450 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:08.109408 (+ 2958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:08.109411 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:08.109413 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:08.109436 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:08.110959 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:08.111566 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:08.111571 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:08.111573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:08.111650 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:08.111993 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:08.111998 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:08.111999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:08.112171 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:08.112275 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:08.112799 (+ 524us) server_negotiation.cc:299] Negotiation successful
0926 16:36:08.112919 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260926 16:36:08.113893 28682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:08.097446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:39439 (local address 127.11.8.194:35783)
0926 16:36:08.098816 (+ 1370us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:08.098888 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:08.100121 (+ 1233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:08.100977 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:08.100986 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:08.102575 (+ 1589us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:36:08.103319 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:08.103336 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:08.104194 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:08.104201 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:08.105927 (+ 1726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:08.105943 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:08.106519 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:08.106527 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:08.106778 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:08.107437 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:36:08.107466 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:36:08.109231 (+ 1765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:36:08.111072 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:08.111077 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:08.111091 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:08.111464 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:08.111741 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:08.111746 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:08.111748 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:08.111892 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:08.112306 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:36:08.112312 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:36:08.112572 (+ 260us) client_negotiation.cc:769] Sending connection context
0926 16:36:08.112759 (+ 187us) client_negotiation.cc:240] Negotiation successful
0926 16:36:08.113121 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1118,"thread_start_us":149,"threads_started":1}
I20260926 16:36:08.115187 28680 heartbeater.cc:347] Connected to a master server at 127.11.8.254:39439
I20260926 16:36:08.115442 28680 heartbeater.cc:464] Registering TS with master...
I20260926 16:36:08.115950 28680 heartbeater.cc:511] Master 127.11.8.254:39439 requested a full tablet report, sending...
I20260926 16:36:08.117055 28355 ts_manager.cc:194] Registered new tserver with Master: 16f6725aa1824e67b3d8f0e5c1bb84b4 (127.11.8.194:38085)
I20260926 16:36:08.117781 28355 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.194@KRBTEST.COM'} at 127.11.8.194:35783
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab.
Entry for principal kudu/127.11.8.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab.
I20260926 16:36:08.129603 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.195:0
--local_ip_for_outbound_sockets=127.11.8.195
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:39439
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:39349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:36:08.238473 28685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:08.238776 28685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:08.238854 28685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:08.242799 28685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:08.242854 28685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:08.242910 28685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.195
I20260926 16:36:08.247880 28685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:39349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:39439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.195
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:08.248739 28685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:08.249629 28685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:08.258718 28690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:08.258823 28691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:08.258853 28693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:08.258925 28685 server_base.cc:1034] running on GCE node
I20260926 16:36:08.259415 28685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:08.260026 28685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:08.261211 28685 hybrid_clock.cc:648] HybridClock initialized: now 1790440568261201 us; error 29 us; skew 500 ppm
Sep 26 16:36:08 dist-test-slave-2zvq krb5kdc[28310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440568, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:36:08.263952 28685 init.cc:377] Logged in from keytab as kudu/127.11.8.195@KRBTEST.COM (short username kudu)
I20260926 16:36:08.264771 28685 webserver.cc:460] Webserver started at http://127.11.8.195:38223/ using document root <none> and password file <none>
I20260926 16:36:08.265311 28685 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:08.265367 28685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:08.265586 28685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:08.267555 28685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-2/data/instance:
uuid: "8ee9d10ba98f4fc1900e5693a3b3664f"
format_stamp: "Formatted at 2026-09-26 16:36:08 on dist-test-slave-2zvq"
server_key: "9a8f5dd65c167d5b922124d6292dce3f"
server_key_iv: "ea580204c8f06f232ee62f0df5703a89"
server_key_version: "encryptionkey@0"
I20260926 16:36:08.268092 28685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance:
uuid: "8ee9d10ba98f4fc1900e5693a3b3664f"
format_stamp: "Formatted at 2026-09-26 16:36:08 on dist-test-slave-2zvq"
server_key: "9a8f5dd65c167d5b922124d6292dce3f"
server_key_iv: "ea580204c8f06f232ee62f0df5703a89"
server_key_version: "encryptionkey@0"
I20260926 16:36:08.271620 28685 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260926 16:36:08.273895 28700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:08.275202 28685 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:36:08.275359 28685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-2/wal
uuid: "8ee9d10ba98f4fc1900e5693a3b3664f"
format_stamp: "Formatted at 2026-09-26 16:36:08 on dist-test-slave-2zvq"
server_key: "9a8f5dd65c167d5b922124d6292dce3f"
server_key_iv: "ea580204c8f06f232ee62f0df5703a89"
server_key_version: "encryptionkey@0"
I20260926 16:36:08.275458 28685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:08.308117 28685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:08.311412 28685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:08.311561 28685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:08.312078 28685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:36:08.312837 28685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:36:08.312885 28685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:08.312925 28685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:36:08.312940 28685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:08.321833 28685 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.195:44143
I20260926 16:36:08.321866 28813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.195:44143 every 8 connection(s)
I20260926 16:36:08.322700 28685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
I20260926 16:36:08.325827 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 28685
I20260926 16:36:08.325992 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance
I20260926 16:36:08.326406 11299 external_mini_cluster.cc:1428] Setting key b0a577fc763c5771b80b0efc0307e415
Sep 26 16:36:08 dist-test-slave-2zvq krb5kdc[28310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440568, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:36:08.336694 28394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:08.324470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:38863 (local address 127.11.8.254:39439)
0926 16:36:08.324637 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:08.324642 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:08.325771 (+ 1129us) server_negotiation.cc:407] Connection header received
0926 16:36:08.326972 (+ 1201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:08.326976 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:08.327029 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:08.327137 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:08.328655 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:08.329256 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:08.330615 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:08.330838 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:08.330928 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:08.333308 (+ 2380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:08.333311 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:08.333313 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:08.333335 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:08.334967 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:08.335419 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:08.335421 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:08.335422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:08.335495 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:08.335785 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:08.335788 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:08.335789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:08.335946 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:08.336027 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:08.336482 (+ 455us) server_negotiation.cc:299] Negotiation successful
0926 16:36:08.336579 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 16:36:08.337430 28816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:08.324948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:39439 (local address 127.11.8.195:38863)
0926 16:36:08.325617 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:08.325657 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:08.326721 (+ 1064us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:08.327355 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:08.327366 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:08.327857 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:36:08.328494 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:08.328515 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:08.329371 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:08.329374 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:08.330485 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:08.330495 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:08.330966 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:08.330969 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:08.331112 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:08.331652 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:36:08.331668 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:36:08.333123 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:36:08.335065 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:08.335070 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:08.335081 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:08.335329 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:08.335574 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:08.335576 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:08.335578 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:08.335708 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:08.336034 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:36:08.336038 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:36:08.336241 (+ 203us) client_negotiation.cc:769] Sending connection context
0926 16:36:08.336402 (+ 161us) client_negotiation.cc:240] Negotiation successful
0926 16:36:08.336637 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":116,"threads_started":1}
I20260926 16:36:08.338673 28814 heartbeater.cc:347] Connected to a master server at 127.11.8.254:39439
I20260926 16:36:08.338929 28814 heartbeater.cc:464] Registering TS with master...
I20260926 16:36:08.339433 28814 heartbeater.cc:511] Master 127.11.8.254:39439 requested a full tablet report, sending...
I20260926 16:36:08.340440 28355 ts_manager.cc:194] Registered new tserver with Master: 8ee9d10ba98f4fc1900e5693a3b3664f (127.11.8.195:44143)
I20260926 16:36:08.340993 28355 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.195@KRBTEST.COM'} at 127.11.8.195:38863
I20260926 16:36:08.352006 11299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 16:36:08.361034 28394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:08.353942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35870 (local address 127.11.8.254:39439)
0926 16:36:08.354090 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:08.354095 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:08.354329 (+ 234us) server_negotiation.cc:407] Connection header received
0926 16:36:08.354482 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:08.354484 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:08.354521 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:08.354587 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:08.355265 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:08.355772 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:08.356579 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:08.356781 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:08.356858 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:08.358033 (+ 1175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:08.358036 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:08.358038 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:08.358061 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:08.359522 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:08.359909 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:08.359912 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:08.359913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:08.359980 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:08.360249 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:08.360251 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:08.360252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:08.360465 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:08.360566 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:08.360775 (+ 209us) server_negotiation.cc:299] Negotiation successful
0926 16:36:08.360856 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 16:36:08.363940 28355 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35870:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:36:08.366989 28355 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:36:08.380739 28828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:08.374933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:38085 (local address 127.0.0.1:39492)
0926 16:36:08.375784 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:08.375797 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:08.375890 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:08.376604 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:08.376607 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:08.376625 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:36:08.376714 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:08.376720 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:08.378178 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:08.378181 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:08.379428 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:08.379438 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:08.380336 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:08.380339 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:08.380446 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:08.380472 (+ 26us) client_negotiation.cc:769] Sending connection context
0926 16:36:08.380512 (+ 40us) client_negotiation.cc:240] Negotiation successful
0926 16:36:08.380597 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":792,"mutex_wait_us":48,"thread_start_us":87,"threads_started":1}
I20260926 16:36:08.380765 28827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:08.374973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38594 (local address 127.11.8.193:42535)
0926 16:36:08.375453 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:08.375472 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:08.375496 (+ 24us) server_negotiation.cc:407] Connection header received
0926 16:36:08.375575 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:08.375580 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:08.375807 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:08.375987 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:36:08.376516 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:08.377584 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:08.379164 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:08.379983 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:08.380184 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:08.380329 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:08.380514 (+ 185us) server_negotiation.cc:299] Negotiation successful
0926 16:36:08.380610 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":76,"threads_started":1}
I20260926 16:36:08.380934 28825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:08.375498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:44143 (local address 127.0.0.1:54022)
0926 16:36:08.376122 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:08.376134 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:08.376222 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:08.376860 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:08.376862 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:08.376877 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:36:08.376961 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:08.376966 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:08.378304 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:08.378308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:08.379465 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:08.379474 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:08.380564 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:08.380568 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:08.380660 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:08.380683 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 16:36:08.380731 (+ 48us) client_negotiation.cc:240] Negotiation successful
0926 16:36:08.380806 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"mutex_wait_us":28,"spinlock_wait_cycles":4224,"thread_start_us":88,"threads_started":1}
I20260926 16:36:08.381017 28830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:08.375777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39492 (local address 127.11.8.194:38085)
0926 16:36:08.376171 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:08.376183 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:08.376203 (+ 20us) server_negotiation.cc:407] Connection header received
0926 16:36:08.376274 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:08.376280 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:08.376461 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:08.376613 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:36:08.376819 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:08.378015 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:08.379551 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:08.380210 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:08.380589 (+ 379us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:08.380717 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:08.380766 (+ 49us) server_negotiation.cc:299] Negotiation successful
0926 16:36:08.380863 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":86,"threads_started":1}
I20260926 16:36:08.381414 28826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:08.374842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:42535 (local address 127.0.0.1:38594)
0926 16:36:08.375273 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:08.375317 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:08.375448 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:08.376171 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:08.376178 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:08.376213 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:36:08.376342 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:08.376352 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:08.377706 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:08.377712 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:08.379035 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:08.379045 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:08.380188 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:08.380191 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:08.380318 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:08.380359 (+ 41us) client_negotiation.cc:769] Sending connection context
0926 16:36:08.380466 (+ 107us) client_negotiation.cc:240] Negotiation successful
0926 16:36:08.381294 (+ 828us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"spinlock_wait_cycles":8192,"thread_start_us":68,"threads_started":1}
I20260926 16:36:08.381553 28829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:08.375647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54022 (local address 127.11.8.195:44143)
0926 16:36:08.376476 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:08.376487 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:08.376509 (+ 22us) server_negotiation.cc:407] Connection header received
0926 16:36:08.376580 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:08.376585 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:08.376747 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:08.376891 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:36:08.377035 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:08.378127 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:08.379590 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:08.380403 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:08.381160 (+ 757us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:08.381291 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:08.381344 (+ 53us) server_negotiation.cc:299] Negotiation successful
0926 16:36:08.381406 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":723,"thread_start_us":132,"threads_started":1}
I20260926 16:36:08.383893 28480 tablet_service.cc:1467] Processing CreateTablet for tablet 9d049adda9614019b25ab8256e358f97 (DEFAULT_TABLE table=test-table [id=19cc6131ad9b4113a2af115499ad4fd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:36:08.383939 28748 tablet_service.cc:1467] Processing CreateTablet for tablet 9d049adda9614019b25ab8256e358f97 (DEFAULT_TABLE table=test-table [id=19cc6131ad9b4113a2af115499ad4fd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:36:08.383939 28614 tablet_service.cc:1467] Processing CreateTablet for tablet 9d049adda9614019b25ab8256e358f97 (DEFAULT_TABLE table=test-table [id=19cc6131ad9b4113a2af115499ad4fd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:36:08.384876 28480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d049adda9614019b25ab8256e358f97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:08.384899 28614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d049adda9614019b25ab8256e358f97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:08.384919 28748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d049adda9614019b25ab8256e358f97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:08.391263 28831 tablet_bootstrap.cc:492] T 9d049adda9614019b25ab8256e358f97 P 3b4e066ad2d5493697f06b41e2e08066: Bootstrap starting.
I20260926 16:36:08.391861 28833 tablet_bootstrap.cc:492] T 9d049adda9614019b25ab8256e358f97 P 16f6725aa1824e67b3d8f0e5c1bb84b4: Bootstrap starting.
I20260926 16:36:08.392524 28832 tablet_bootstrap.cc:492] T 9d049adda9614019b25ab8256e358f97 P 8ee9d10ba98f4fc1900e5693a3b3664f: Bootstrap starting.
I20260926 16:36:08.393545 28831 tablet_bootstrap.cc:654] T 9d049adda9614019b25ab8256e358f97 P 3b4e066ad2d5493697f06b41e2e08066: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:08.394157 28832 tablet_bootstrap.cc:654] T 9d049adda9614019b25ab8256e358f97 P 8ee9d10ba98f4fc1900e5693a3b3664f: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:08.394352 28833 tablet_bootstrap.cc:654] T 9d049adda9614019b25ab8256e358f97 P 16f6725aa1824e67b3d8f0e5c1bb84b4: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:08.394923 28831 log.cc:824] T 9d049adda9614019b25ab8256e358f97 P 3b4e066ad2d5493697f06b41e2e08066: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:08.395079 28833 log.cc:824] T 9d049adda9614019b25ab8256e358f97 P 16f6725aa1824e67b3d8f0e5c1bb84b4: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:08.395336 28832 log.cc:824] T 9d049adda9614019b25ab8256e358f97 P 8ee9d10ba98f4fc1900e5693a3b3664f: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:08.396957 28831 tablet_bootstrap.cc:492] T 9d049adda9614019b25ab8256e358f97 P 3b4e066ad2d5493697f06b41e2e08066: No bootstrap required, opened a new log
I20260926 16:36:08.396960 28833 tablet_bootstrap.cc:492] T 9d049adda9614019b25ab8256e358f97 P 16f6725aa1824e67b3d8f0e5c1bb84b4: No bootstrap required, opened a new log
I20260926 16:36:08.397145 28833 ts_tablet_manager.cc:1397] T 9d049adda9614019b25ab8256e358f97 P 16f6725aa1824e67b3d8f0e5c1bb84b4: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260926 16:36:08.397145 28831 ts_tablet_manager.cc:1397] T 9d049adda9614019b25ab8256e358f97 P 3b4e066ad2d5493697f06b41e2e08066: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260926 16:36:08.397262 28832 tablet_bootstrap.cc:492] T 9d049adda9614019b25ab8256e358f97 P 8ee9d10ba98f4fc1900e5693a3b3664f: No bootstrap required, opened a new log
I20260926 16:36:08.397436 28832 ts_tablet_manager.cc:1397] T 9d049adda9614019b25ab8256e358f97 P 8ee9d10ba98f4fc1900e5693a3b3664f: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260926 16:36:08.400279 28832 raft_consensus.cc:348] T 9d049adda9614019b25ab8256e358f97 P 8ee9d10ba98f4fc1900e5693a3b3664f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b4e066ad2d5493697f06b41e2e08066" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42535 } } peers { permanent_uuid: "16f6725aa1824e67b3d8f0e5c1bb84b4" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 38085 } } peers { permanent_uuid: "8ee9d10ba98f4fc1900e5693a3b3664f" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 44143 } }
I20260926 16:36:08.400519 28832 raft_consensus.cc:374] T 9d049adda9614019b25ab8256e358f97 P 8ee9d10ba98f4fc1900e5693a3b3664f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:36:08.400600 28832 raft_consensus.cc:729] T 9d049adda9614019b25ab8256e358f97 P 8ee9d10ba98f4fc1900e5693a3b3664f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ee9d10ba98f4fc1900e5693a3b3664f, State: Initialized, Role: FOLLOWER
I20260926 16:36:08.401041 28832 consensus_queue.cc:260] T 9d049adda9614019b25ab8256e358f97 P 8ee9d10ba98f4fc1900e5693a3b3664f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b4e066ad2d5493697f06b41e2e08066" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42535 } } peers { permanent_uuid: "16f6725aa1824e67b3d8f0e5c1bb84b4" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 38085 } } peers { permanent_uuid: "8ee9d10ba98f4fc1900e5693a3b3664f" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 44143 } }
I20260926 16:36:08.400952 28831 raft_consensus.cc:348] T 9d049adda9614019b25ab8256e358f97 P 3b4e066ad2d5493697f06b41e2e08066 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b4e066ad2d5493697f06b41e2e08066" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42535 } } peers { permanent_uuid: "16f6725aa1824e67b3d8f0e5c1bb84b4" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 38085 } } peers { permanent_uuid: "8ee9d10ba98f4fc1900e5693a3b3664f" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 44143 } }
I20260926 16:36:08.400954 28833 raft_consensus.cc:348] T 9d049adda9614019b25ab8256e358f97 P 16f6725aa1824e67b3d8f0e5c1bb84b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b4e066ad2d5493697f06b41e2e08066" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42535 } } peers { permanent_uuid: "16f6725aa1824e67b3d8f0e5c1bb84b4" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 38085 } } peers { permanent_uuid: "8ee9d10ba98f4fc1900e5693a3b3664f" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 44143 } }
I20260926 16:36:08.401180 28831 raft_consensus.cc:374] T 9d049adda9614019b25ab8256e358f97 P 3b4e066ad2d5493697f06b41e2e08066 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:36:08.401180 28833 raft_consensus.cc:374] T 9d049adda9614019b25ab8256e358f97 P 16f6725aa1824e67b3d8f0e5c1bb84b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:36:08.401237 28833 raft_consensus.cc:729] T 9d049adda9614019b25ab8256e358f97 P 16f6725aa1824e67b3d8f0e5c1bb84b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16f6725aa1824e67b3d8f0e5c1bb84b4, State: Initialized, Role: FOLLOWER
I20260926 16:36:08.401460 28831 raft_consensus.cc:729] T 9d049adda9614019b25ab8256e358f97 P 3b4e066ad2d5493697f06b41e2e08066 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b4e066ad2d5493697f06b41e2e08066, State: Initialized, Role: FOLLOWER
I20260926 16:36:08.401697 28814 heartbeater.cc:503] Master 127.11.8.254:39439 was elected leader, sending a full tablet report...
I20260926 16:36:08.401654 28833 consensus_queue.cc:260] T 9d049adda9614019b25ab8256e358f97 P 16f6725aa1824e67b3d8f0e5c1bb84b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b4e066ad2d5493697f06b41e2e08066" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42535 } } peers { permanent_uuid: "16f6725aa1824e67b3d8f0e5c1bb84b4" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 38085 } } peers { permanent_uuid: "8ee9d10ba98f4fc1900e5693a3b3664f" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 44143 } }
I20260926 16:36:08.402066 28832 ts_tablet_manager.cc:1428] T 9d049adda9614019b25ab8256e358f97 P 8ee9d10ba98f4fc1900e5693a3b3664f: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260926 16:36:08.402355 28680 heartbeater.cc:503] Master 127.11.8.254:39439 was elected leader, sending a full tablet report...
I20260926 16:36:08.402554 28833 ts_tablet_manager.cc:1428] T 9d049adda9614019b25ab8256e358f97 P 16f6725aa1824e67b3d8f0e5c1bb84b4: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260926 16:36:08.403124 28831 consensus_queue.cc:260] T 9d049adda9614019b25ab8256e358f97 P 3b4e066ad2d5493697f06b41e2e08066 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b4e066ad2d5493697f06b41e2e08066" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42535 } } peers { permanent_uuid: "16f6725aa1824e67b3d8f0e5c1bb84b4" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 38085 } } peers { permanent_uuid: "8ee9d10ba98f4fc1900e5693a3b3664f" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 44143 } }
I20260926 16:36:08.403877 28546 heartbeater.cc:503] Master 127.11.8.254:39439 was elected leader, sending a full tablet report...
I20260926 16:36:08.403966 28831 ts_tablet_manager.cc:1428] T 9d049adda9614019b25ab8256e358f97 P 3b4e066ad2d5493697f06b41e2e08066: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260926 16:36:08.442875 28838 raft_consensus.cc:482] T 9d049adda9614019b25ab8256e358f97 P 16f6725aa1824e67b3d8f0e5c1bb84b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:36:08.443112 28838 raft_consensus.cc:504] T 9d049adda9614019b25ab8256e358f97 P 16f6725aa1824e67b3d8f0e5c1bb84b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b4e066ad2d5493697f06b41e2e08066" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42535 } } peers { permanent_uuid: "16f6725aa1824e67b3d8f0e5c1bb84b4" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 38085 } } peers { permanent_uuid: "8ee9d10ba98f4fc1900e5693a3b3664f" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 44143 } }
I20260926 16:36:08.444217 28838 leader_election.cc:288] T 9d049adda9614019b25ab8256e358f97 P 16f6725aa1824e67b3d8f0e5c1bb84b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b4e066ad2d5493697f06b41e2e08066 (127.11.8.193:42535), 8ee9d10ba98f4fc1900e5693a3b3664f (127.11.8.195:44143)
I20260926 16:36:08.448102 28829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:08.444542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:34087 (local address 127.11.8.195:44143)
0926 16:36:08.444668 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:08.444673 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:08.444755 (+ 82us) server_negotiation.cc:407] Connection header received
0926 16:36:08.444905 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:08.444910 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:08.444974 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:08.445053 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:36:08.445352 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:08.445931 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:08.447070 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:08.447506 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:08.447629 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:08.447679 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:08.447911 (+ 232us) server_negotiation.cc:299] Negotiation successful
0926 16:36:08.447975 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 16:36:08.448118 28682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:08.444457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:44143 (local address 127.11.8.194:34087)
0926 16:36:08.444669 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:08.444685 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:08.444788 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:08.445085 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:08.445089 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:08.445106 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:36:08.445241 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:08.445248 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:08.446035 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:08.446038 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:08.446952 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:08.446961 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:08.447622 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:08.447626 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:08.447731 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:08.447800 (+ 69us) client_negotiation.cc:769] Sending connection context
0926 16:36:08.447889 (+ 89us) client_negotiation.cc:240] Negotiation successful
0926 16:36:08.447977 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":42}
I20260926 16:36:08.448107 28827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:08.444555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:37107 (local address 127.11.8.193:42535)
0926 16:36:08.444668 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:08.444673 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:08.444993 (+ 320us) server_negotiation.cc:407] Connection header received
0926 16:36:08.445115 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:08.445118 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:08.445167 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:08.445246 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:36:08.445540 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:08.446049 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:08.447136 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:08.447520 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:08.447652 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:08.447704 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:08.447915 (+ 211us) server_negotiation.cc:299] Negotiation successful
0926 16:36:08.447981 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 16:36:08.448137 28840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:08.444462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:42535 (local address 127.11.8.194:37107)
0926 16:36:08.444893 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:08.444907 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:08.445013 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:08.445271 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:08.445273 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:08.445287 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:36:08.445430 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:08.445436 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:08.446154 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:08.446156 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:08.446994 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:08.447024 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:08.447655 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:08.447659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:08.447784 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:08.447804 (+ 20us) client_negotiation.cc:769] Sending connection context
0926 16:36:08.447885 (+ 81us) client_negotiation.cc:240] Negotiation successful
0926 16:36:08.447955 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"mutex_wait_us":107,"thread_start_us":112,"threads_started":1}
I20260926 16:36:08.448711 28768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d049adda9614019b25ab8256e358f97" candidate_uuid: "16f6725aa1824e67b3d8f0e5c1bb84b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ee9d10ba98f4fc1900e5693a3b3664f" is_pre_election: true
I20260926 16:36:08.448738 28500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d049adda9614019b25ab8256e358f97" candidate_uuid: "16f6725aa1824e67b3d8f0e5c1bb84b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b4e066ad2d5493697f06b41e2e08066" is_pre_election: true
I20260926 16:36:08.448997 28500 raft_consensus.cc:2454] T 9d049adda9614019b25ab8256e358f97 P 3b4e066ad2d5493697f06b41e2e08066 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16f6725aa1824e67b3d8f0e5c1bb84b4 in term 0.
I20260926 16:36:08.448997 28768 raft_consensus.cc:2454] T 9d049adda9614019b25ab8256e358f97 P 8ee9d10ba98f4fc1900e5693a3b3664f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16f6725aa1824e67b3d8f0e5c1bb84b4 in term 0.
I20260926 16:36:08.449493 28568 leader_election.cc:302] T 9d049adda9614019b25ab8256e358f97 P 16f6725aa1824e67b3d8f0e5c1bb84b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 16f6725aa1824e67b3d8f0e5c1bb84b4, 3b4e066ad2d5493697f06b41e2e08066, 8ee9d10ba98f4fc1900e5693a3b3664f; no voters:
I20260926 16:36:08.449744 28838 raft_consensus.cc:2781] T 9d049adda9614019b25ab8256e358f97 P 16f6725aa1824e67b3d8f0e5c1bb84b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:36:08.449821 28838 raft_consensus.cc:482] T 9d049adda9614019b25ab8256e358f97 P 16f6725aa1824e67b3d8f0e5c1bb84b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:36:08.449862 28838 raft_consensus.cc:3037] T 9d049adda9614019b25ab8256e358f97 P 16f6725aa1824e67b3d8f0e5c1bb84b4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:08.450912 28838 raft_consensus.cc:504] T 9d049adda9614019b25ab8256e358f97 P 16f6725aa1824e67b3d8f0e5c1bb84b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b4e066ad2d5493697f06b41e2e08066" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42535 } } peers { permanent_uuid: "16f6725aa1824e67b3d8f0e5c1bb84b4" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 38085 } } peers { permanent_uuid: "8ee9d10ba98f4fc1900e5693a3b3664f" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 44143 } }
I20260926 16:36:08.451277 28838 leader_election.cc:288] T 9d049adda9614019b25ab8256e358f97 P 16f6725aa1824e67b3d8f0e5c1bb84b4 [CANDIDATE]: Term 1 election: Requested vote from peers 3b4e066ad2d5493697f06b41e2e08066 (127.11.8.193:42535), 8ee9d10ba98f4fc1900e5693a3b3664f (127.11.8.195:44143)
I20260926 16:36:08.451581 28500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d049adda9614019b25ab8256e358f97" candidate_uuid: "16f6725aa1824e67b3d8f0e5c1bb84b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b4e066ad2d5493697f06b41e2e08066"
I20260926 16:36:08.451632 28768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d049adda9614019b25ab8256e358f97" candidate_uuid: "16f6725aa1824e67b3d8f0e5c1bb84b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ee9d10ba98f4fc1900e5693a3b3664f"
I20260926 16:36:08.451697 28500 raft_consensus.cc:3037] T 9d049adda9614019b25ab8256e358f97 P 3b4e066ad2d5493697f06b41e2e08066 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:08.451730 28768 raft_consensus.cc:3037] T 9d049adda9614019b25ab8256e358f97 P 8ee9d10ba98f4fc1900e5693a3b3664f [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:08.452996 28500 raft_consensus.cc:2454] T 9d049adda9614019b25ab8256e358f97 P 3b4e066ad2d5493697f06b41e2e08066 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16f6725aa1824e67b3d8f0e5c1bb84b4 in term 1.
I20260926 16:36:08.452996 28768 raft_consensus.cc:2454] T 9d049adda9614019b25ab8256e358f97 P 8ee9d10ba98f4fc1900e5693a3b3664f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16f6725aa1824e67b3d8f0e5c1bb84b4 in term 1.
I20260926 16:36:08.453281 28569 leader_election.cc:302] T 9d049adda9614019b25ab8256e358f97 P 16f6725aa1824e67b3d8f0e5c1bb84b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16f6725aa1824e67b3d8f0e5c1bb84b4, 8ee9d10ba98f4fc1900e5693a3b3664f; no voters:
I20260926 16:36:08.453438 28838 raft_consensus.cc:2781] T 9d049adda9614019b25ab8256e358f97 P 16f6725aa1824e67b3d8f0e5c1bb84b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:36:08.453653 28838 raft_consensus.cc:686] T 9d049adda9614019b25ab8256e358f97 P 16f6725aa1824e67b3d8f0e5c1bb84b4 [term 1 LEADER]: Becoming Leader. State: Replica: 16f6725aa1824e67b3d8f0e5c1bb84b4, State: Running, Role: LEADER
I20260926 16:36:08.453965 28838 consensus_queue.cc:237] T 9d049adda9614019b25ab8256e358f97 P 16f6725aa1824e67b3d8f0e5c1bb84b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b4e066ad2d5493697f06b41e2e08066" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42535 } } peers { permanent_uuid: "16f6725aa1824e67b3d8f0e5c1bb84b4" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 38085 } } peers { permanent_uuid: "8ee9d10ba98f4fc1900e5693a3b3664f" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 44143 } }
I20260926 16:36:08.456888 28354 catalog_manager.cc:5495] T 9d049adda9614019b25ab8256e358f97 P 16f6725aa1824e67b3d8f0e5c1bb84b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 16f6725aa1824e67b3d8f0e5c1bb84b4 (127.11.8.194). New cstate: current_term: 1 leader_uuid: "16f6725aa1824e67b3d8f0e5c1bb84b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b4e066ad2d5493697f06b41e2e08066" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16f6725aa1824e67b3d8f0e5c1bb84b4" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 38085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ee9d10ba98f4fc1900e5693a3b3664f" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 44143 } health_report { overall_health: UNKNOWN } } }
I20260926 16:36:08.477145 28830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:08.473461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39500 (local address 127.11.8.194:38085)
0926 16:36:08.473587 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:08.473593 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:08.473612 (+ 19us) server_negotiation.cc:407] Connection header received
0926 16:36:08.473800 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:08.473803 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:08.473852 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:08.473960 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:36:08.474201 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:08.474757 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:08.475591 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:08.475808 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:08.475914 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:08.476310 (+ 396us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:36:08.476740 (+ 430us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:36:08.476835 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:08.476890 (+ 55us) server_negotiation.cc:299] Negotiation successful
0926 16:36:08.476943 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 16:36:08.482851 28500 raft_consensus.cc:1261] T 9d049adda9614019b25ab8256e358f97 P 3b4e066ad2d5493697f06b41e2e08066 [term 1 FOLLOWER]: Refusing update from remote peer 16f6725aa1824e67b3d8f0e5c1bb84b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:36:08.482851 28768 raft_consensus.cc:1261] T 9d049adda9614019b25ab8256e358f97 P 8ee9d10ba98f4fc1900e5693a3b3664f [term 1 FOLLOWER]: Refusing update from remote peer 16f6725aa1824e67b3d8f0e5c1bb84b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:36:08.483520 28838 consensus_queue.cc:1035] T 9d049adda9614019b25ab8256e358f97 P 16f6725aa1824e67b3d8f0e5c1bb84b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ee9d10ba98f4fc1900e5693a3b3664f" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 44143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:36:08.483681 28841 consensus_queue.cc:1035] T 9d049adda9614019b25ab8256e358f97 P 16f6725aa1824e67b3d8f0e5c1bb84b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b4e066ad2d5493697f06b41e2e08066" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:36:08.492875 28845 mvcc.cc:205] Tried to move back new op lower bound from 7333644568498925568 to 7333644568390356992. Current Snapshot: MvccSnapshot[applied={T|T < 7333644568498925568}]
I20260926 16:36:08.497874 28354 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:36:08.498018 28354 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:36:08.500339 28354 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 999b5b77354046e6bbc58955d32b6c9e: Sending DeleteTablet for 3 replicas of tablet 9d049adda9614019b25ab8256e358f97
I20260926 16:36:08.501058 28480 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d049adda9614019b25ab8256e358f97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:36:08 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:38594
I20260926 16:36:08.501099 28748 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d049adda9614019b25ab8256e358f97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:36:08 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:54022
I20260926 16:36:08.501538 28854 tablet_replica.cc:330] T 9d049adda9614019b25ab8256e358f97 P 3b4e066ad2d5493697f06b41e2e08066: stopping tablet replica
I20260926 16:36:08.501613 28614 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d049adda9614019b25ab8256e358f97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:36:08 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:39492
I20260926 16:36:08.501847 28854 raft_consensus.cc:2229] T 9d049adda9614019b25ab8256e358f97 P 3b4e066ad2d5493697f06b41e2e08066 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:36:08.502063 28856 tablet_replica.cc:330] T 9d049adda9614019b25ab8256e358f97 P 16f6725aa1824e67b3d8f0e5c1bb84b4: stopping tablet replica
I20260926 16:36:08.502151 28854 raft_consensus.cc:2258] T 9d049adda9614019b25ab8256e358f97 P 3b4e066ad2d5493697f06b41e2e08066 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:36:08.502471 28856 raft_consensus.cc:2229] T 9d049adda9614019b25ab8256e358f97 P 16f6725aa1824e67b3d8f0e5c1bb84b4 [term 1 LEADER]: Raft consensus shutting down.
I20260926 16:36:08.502645 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 28417
I20260926 16:36:08.503700 28855 tablet_replica.cc:330] T 9d049adda9614019b25ab8256e358f97 P 8ee9d10ba98f4fc1900e5693a3b3664f: stopping tablet replica
I20260926 16:36:08.503963 28855 raft_consensus.cc:2229] T 9d049adda9614019b25ab8256e358f97 P 8ee9d10ba98f4fc1900e5693a3b3664f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:36:08.504237 28855 raft_consensus.cc:2258] T 9d049adda9614019b25ab8256e358f97 P 8ee9d10ba98f4fc1900e5693a3b3664f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:36:08.505400 28856 raft_consensus.cc:2258] T 9d049adda9614019b25ab8256e358f97 P 16f6725aa1824e67b3d8f0e5c1bb84b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:36:08.505757 28855 ts_tablet_manager.cc:1905] T 9d049adda9614019b25ab8256e358f97 P 8ee9d10ba98f4fc1900e5693a3b3664f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:36:08.506875 28856 ts_tablet_manager.cc:1905] T 9d049adda9614019b25ab8256e358f97 P 16f6725aa1824e67b3d8f0e5c1bb84b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:36:08.509124 28855 ts_tablet_manager.cc:1918] T 9d049adda9614019b25ab8256e358f97 P 8ee9d10ba98f4fc1900e5693a3b3664f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:36:08.509227 28855 log.cc:1196] T 9d049adda9614019b25ab8256e358f97 P 8ee9d10ba98f4fc1900e5693a3b3664f: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-2/wal/wals/9d049adda9614019b25ab8256e358f97
I20260926 16:36:08.509577 28855 ts_tablet_manager.cc:1939] T 9d049adda9614019b25ab8256e358f97 P 8ee9d10ba98f4fc1900e5693a3b3664f: Deleting consensus metadata
I20260926 16:36:08.509737 28856 ts_tablet_manager.cc:1918] T 9d049adda9614019b25ab8256e358f97 P 16f6725aa1824e67b3d8f0e5c1bb84b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:36:08.509874 28856 log.cc:1196] T 9d049adda9614019b25ab8256e358f97 P 16f6725aa1824e67b3d8f0e5c1bb84b4: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790440324823152-11299-0/minicluster-data/ts-1/wal/wals/9d049adda9614019b25ab8256e358f97
W20260926 16:36:08.509912 28340 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.8.193:42535 (error 108)
I20260926 16:36:08.510319 28856 ts_tablet_manager.cc:1939] T 9d049adda9614019b25ab8256e358f97 P 16f6725aa1824e67b3d8f0e5c1bb84b4: Deleting consensus metadata
I20260926 16:36:08.510699 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 28551
I20260926 16:36:08.510793 28341 catalog_manager.cc:4841] TS 8ee9d10ba98f4fc1900e5693a3b3664f (127.11.8.195:44143): tablet 9d049adda9614019b25ab8256e358f97 (table test-table [id=19cc6131ad9b4113a2af115499ad4fd2]) successfully deleted
I20260926 16:36:08.512177 28826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:08.511912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:42535 (local address 127.0.0.1:38602)
0926 16:36:08.512036 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:08.512111 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.193:42535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260926 16:36:08.512383 28340 catalog_manager.cc:4584] TS 3b4e066ad2d5493697f06b41e2e08066 (127.11.8.193:42535): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9d049adda9614019b25ab8256e358f97: Network error: Client connection negotiation failed: client connection to 127.11.8.193:42535: connect: Connection refused (error 111)
I20260926 16:36:08.516738 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 28685
I20260926 16:36:08.517908 28826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:08.516916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:38085 (local address 127.0.0.1:39510)
0926 16:36:08.517756 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:08.517819 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.194:38085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":742}
I20260926 16:36:08.522065 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 28323
2026-09-26T16:36:08Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2265 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-26T16:36:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:36:08Z Disabled control of system clock
I20260926 16:36:08.551208 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:46715
--webserver_interface=127.11.8.254
--webserver_port=0
--builtin_ntp_servers=127.11.8.212:38423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:46715
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 16:36:08.659063 28861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:08.659317 28861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:08.659354 28861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:08.663776 28861 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:36:08.663837 28861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:08.663853 28861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:08.663870 28861 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:36:08.663942 28861 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:36:08.668771 28861 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:38423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:46715
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:46715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28861
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:08.669782 28861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:08.671188 28861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:08.679540 28866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:08.679636 28867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:08.679553 28869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:08.679769 28861 server_base.cc:1034] running on GCE node
I20260926 16:36:08.680392 28861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:08.681463 28861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:08.682686 28861 hybrid_clock.cc:648] HybridClock initialized: now 1790440568682673 us; error 24 us; skew 500 ppm
I20260926 16:36:08.684626 28861 webserver.cc:460] Webserver started at http://127.11.8.254:36237/ using document root <none> and password file <none>
I20260926 16:36:08.685212 28861 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:08.685307 28861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:08.685544 28861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:08.687703 28861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "18f43254405d4605b6484c8be9a1ccf8"
format_stamp: "Formatted at 2026-09-26 16:36:08 on dist-test-slave-2zvq"
server_key: "44725f1ec37eeb44ced8d3dd0c21495f"
server_key_iv: "1563ecfdfcc3558f14e3b516de072c29"
server_key_version: "encryptionkey@0"
I20260926 16:36:08.688253 28861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "18f43254405d4605b6484c8be9a1ccf8"
format_stamp: "Formatted at 2026-09-26 16:36:08 on dist-test-slave-2zvq"
server_key: "44725f1ec37eeb44ced8d3dd0c21495f"
server_key_iv: "1563ecfdfcc3558f14e3b516de072c29"
server_key_version: "encryptionkey@0"
I20260926 16:36:08.691871 28861 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260926 16:36:08.694245 28875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:08.695526 28861 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:36:08.695659 28861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "18f43254405d4605b6484c8be9a1ccf8"
format_stamp: "Formatted at 2026-09-26 16:36:08 on dist-test-slave-2zvq"
server_key: "44725f1ec37eeb44ced8d3dd0c21495f"
server_key_iv: "1563ecfdfcc3558f14e3b516de072c29"
server_key_version: "encryptionkey@0"
I20260926 16:36:08.695762 28861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:08.742190 28861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:08.743022 28861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:08.743217 28861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:08.749948 28861 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:46715
I20260926 16:36:08.750001 28927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:46715 every 8 connection(s)
I20260926 16:36:08.750897 28861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:36:08.753849 28928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:08.757140 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 28861
I20260926 16:36:08.757270 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:36:08.757567 11299 external_mini_cluster.cc:1428] Setting key 6e587534e954c16ee4f2f9f7260b6375
I20260926 16:36:08.759959 28928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18f43254405d4605b6484c8be9a1ccf8: Bootstrap starting.
I20260926 16:36:08.762053 28928 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18f43254405d4605b6484c8be9a1ccf8: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:08.762826 28928 log.cc:824] T 00000000000000000000000000000000 P 18f43254405d4605b6484c8be9a1ccf8: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:08.764554 28928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18f43254405d4605b6484c8be9a1ccf8: No bootstrap required, opened a new log
I20260926 16:36:08.766753 28931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:08.759086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52364 (local address 127.11.8.254:46715)
0926 16:36:08.759736 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:08.759750 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:08.759784 (+ 34us) server_negotiation.cc:407] Connection header received
0926 16:36:08.760464 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:08.760483 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:08.760858 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:08.761182 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:08.761603 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:08.762639 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:08.763552 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:08.764013 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:08.764180 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:08.765010 (+ 830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:08.765015 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:08.765026 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:36:08.765384 (+ 358us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:08.765444 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:36:08.765462 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:08.765569 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:08.765693 (+ 124us) server_negotiation.cc:299] Negotiation successful
0926 16:36:08.765950 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":130,"threads_started":1}
I20260926 16:36:08.768271 28928 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18f43254405d4605b6484c8be9a1ccf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f43254405d4605b6484c8be9a1ccf8" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 46715 } }
I20260926 16:36:08.768534 28928 raft_consensus.cc:374] T 00000000000000000000000000000000 P 18f43254405d4605b6484c8be9a1ccf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:36:08.768580 28928 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18f43254405d4605b6484c8be9a1ccf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18f43254405d4605b6484c8be9a1ccf8, State: Initialized, Role: FOLLOWER
I20260926 16:36:08.769023 28928 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18f43254405d4605b6484c8be9a1ccf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f43254405d4605b6484c8be9a1ccf8" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 46715 } }
I20260926 16:36:08.769146 28928 raft_consensus.cc:388] T 00000000000000000000000000000000 P 18f43254405d4605b6484c8be9a1ccf8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:36:08.769184 28928 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18f43254405d4605b6484c8be9a1ccf8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:36:08.769251 28928 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18f43254405d4605b6484c8be9a1ccf8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:08.770026 28928 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18f43254405d4605b6484c8be9a1ccf8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f43254405d4605b6484c8be9a1ccf8" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 46715 } }
I20260926 16:36:08.770354 28928 leader_election.cc:302] T 00000000000000000000000000000000 P 18f43254405d4605b6484c8be9a1ccf8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 18f43254405d4605b6484c8be9a1ccf8; no voters:
I20260926 16:36:08.770604 28928 leader_election.cc:288] T 00000000000000000000000000000000 P 18f43254405d4605b6484c8be9a1ccf8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:36:08.770722 28933 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18f43254405d4605b6484c8be9a1ccf8 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:36:08.770982 28933 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18f43254405d4605b6484c8be9a1ccf8 [term 1 LEADER]: Becoming Leader. State: Replica: 18f43254405d4605b6484c8be9a1ccf8, State: Running, Role: LEADER
I20260926 16:36:08.771289 28933 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18f43254405d4605b6484c8be9a1ccf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f43254405d4605b6484c8be9a1ccf8" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 46715 } }
I20260926 16:36:08.771883 28928 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18f43254405d4605b6484c8be9a1ccf8 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:36:08.773038 28934 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18f43254405d4605b6484c8be9a1ccf8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "18f43254405d4605b6484c8be9a1ccf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f43254405d4605b6484c8be9a1ccf8" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 46715 } } }
I20260926 16:36:08.773038 28935 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18f43254405d4605b6484c8be9a1ccf8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18f43254405d4605b6484c8be9a1ccf8. Latest consensus state: current_term: 1 leader_uuid: "18f43254405d4605b6484c8be9a1ccf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f43254405d4605b6484c8be9a1ccf8" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 46715 } } }
I20260926 16:36:08.773252 28935 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18f43254405d4605b6484c8be9a1ccf8 [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:08.773211 28934 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18f43254405d4605b6484c8be9a1ccf8 [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:08.773669 28944 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:36:08.776705 28944 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:36:08.781976 28944 catalog_manager.cc:1347] Generated new cluster ID: 74beca3d58804055b3dc0e0bb5e4b2a4
I20260926 16:36:08.782037 28944 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:36:08.797556 28944 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:36:08.798460 28944 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:36:08.806495 28944 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 18f43254405d4605b6484c8be9a1ccf8: Generated new TSK 0
I20260926 16:36:08.807111 28944 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:36:08.818092 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.193:0
--local_ip_for_outbound_sockets=127.11.8.193
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:46715
--builtin_ntp_servers=127.11.8.212:38423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 16:36:08.926540 28952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:08.926771 28952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:08.926806 28952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:08.930707 28952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:08.930750 28952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:08.930846 28952 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.193
I20260926 16:36:08.935364 28952 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:38423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:46715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.193
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:08.936321 28952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:08.937604 28952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:08.946976 28957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:08.946973 28960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:08.947083 28952 server_base.cc:1034] running on GCE node
W20260926 16:36:08.946973 28958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:08.947654 28952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:08.948213 28952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:08.949391 28952 hybrid_clock.cc:648] HybridClock initialized: now 1790440568949380 us; error 23 us; skew 500 ppm
I20260926 16:36:08.951128 28952 webserver.cc:460] Webserver started at http://127.11.8.193:42325/ using document root <none> and password file <none>
I20260926 16:36:08.951669 28952 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:08.951759 28952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:08.951979 28952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:08.953931 28952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-0/data/instance:
uuid: "4fb6e0d14b834c84ac8e6dd8efcc9934"
format_stamp: "Formatted at 2026-09-26 16:36:08 on dist-test-slave-2zvq"
server_key: "a7a75bbd3391f731c7ef317bf9a7e98f"
server_key_iv: "8b4a822107ab5ea85602bc57541b4f7d"
server_key_version: "encryptionkey@0"
I20260926 16:36:08.954519 28952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance:
uuid: "4fb6e0d14b834c84ac8e6dd8efcc9934"
format_stamp: "Formatted at 2026-09-26 16:36:08 on dist-test-slave-2zvq"
server_key: "a7a75bbd3391f731c7ef317bf9a7e98f"
server_key_iv: "8b4a822107ab5ea85602bc57541b4f7d"
server_key_version: "encryptionkey@0"
I20260926 16:36:08.957808 28952 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.002s
I20260926 16:36:08.960044 28966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:08.961283 28952 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:36:08.961424 28952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-0/wal
uuid: "4fb6e0d14b834c84ac8e6dd8efcc9934"
format_stamp: "Formatted at 2026-09-26 16:36:08 on dist-test-slave-2zvq"
server_key: "a7a75bbd3391f731c7ef317bf9a7e98f"
server_key_iv: "8b4a822107ab5ea85602bc57541b4f7d"
server_key_version: "encryptionkey@0"
I20260926 16:36:08.961529 28952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:08.985285 28952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:08.985915 28952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:08.986102 28952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:08.986678 28952 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:36:08.987548 28952 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:36:08.987618 28952 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:08.987684 28952 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:36:08.987731 28952 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:08.996459 28952 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.193:37929
I20260926 16:36:08.996506 29079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.193:37929 every 8 connection(s)
I20260926 16:36:08.997339 28952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
I20260926 16:36:09.003968 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 28952
I20260926 16:36:09.004108 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance
I20260926 16:36:09.004526 11299 external_mini_cluster.cc:1428] Setting key 8d8d719719bbdd1bedc51b51d38dc3a5
I20260926 16:36:09.006713 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.194:0
--local_ip_for_outbound_sockets=127.11.8.194
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:46715
--builtin_ntp_servers=127.11.8.212:38423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 16:36:09.007948 28931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:08.999595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:35553 (local address 127.11.8.254:46715)
0926 16:36:08.999735 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:08.999740 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:09.000813 (+ 1073us) server_negotiation.cc:407] Connection header received
0926 16:36:09.002147 (+ 1334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:09.002149 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:09.002193 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:09.002284 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:09.003454 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:09.003980 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:09.005681 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:09.006012 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:09.006094 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:09.007495 (+ 1401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:09.007498 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:09.007500 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:36:09.007511 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:09.007529 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:36:09.007537 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:09.007595 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:09.007795 (+ 200us) server_negotiation.cc:299] Negotiation successful
0926 16:36:09.007839 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 16:36:09.008960 29082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:09.000007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46715 (local address 127.11.8.193:35553)
0926 16:36:09.000654 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:09.000694 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:09.001958 (+ 1264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:09.002557 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:09.002576 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:09.002705 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:36:09.003296 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:09.003312 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:09.004148 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:09.004152 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:09.005558 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:09.005569 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:09.006172 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:09.006177 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:09.006418 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:09.007178 (+ 760us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:36:09.007183 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:36:09.007246 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:36:09.007338 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:36:09.007625 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:36:09.007638 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:36:09.007649 (+ 11us) client_negotiation.cc:769] Sending connection context
0926 16:36:09.007795 (+ 146us) client_negotiation.cc:240] Negotiation successful
0926 16:36:09.008112 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":98,"threads_started":1}
I20260926 16:36:09.010334 29080 heartbeater.cc:347] Connected to a master server at 127.11.8.254:46715
I20260926 16:36:09.010594 29080 heartbeater.cc:464] Registering TS with master...
I20260926 16:36:09.011140 29080 heartbeater.cc:511] Master 127.11.8.254:46715 requested a full tablet report, sending...
I20260926 16:36:09.012950 28892 ts_manager.cc:194] Registered new tserver with Master: 4fb6e0d14b834c84ac8e6dd8efcc9934 (127.11.8.193:37929)
I20260926 16:36:09.014705 28892 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.8.193:35553
W20260926 16:36:09.114300 29083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:09.114526 29083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:09.114562 29083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:09.118355 29083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:09.118391 29083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:09.118446 29083 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.194
I20260926 16:36:09.122990 29083 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:38423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:46715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.194
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:09.123867 29083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:09.125077 29083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:09.134594 29089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:09.134608 29088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:09.134653 29083 server_base.cc:1034] running on GCE node
W20260926 16:36:09.134609 29091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:09.135195 29083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:09.135882 29083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:09.137105 29083 hybrid_clock.cc:648] HybridClock initialized: now 1790440569137089 us; error 35 us; skew 500 ppm
I20260926 16:36:09.139029 29083 webserver.cc:460] Webserver started at http://127.11.8.194:41615/ using document root <none> and password file <none>
I20260926 16:36:09.139604 29083 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:09.139694 29083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:09.139918 29083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:09.141920 29083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-1/data/instance:
uuid: "35edeb75b9ba4cb3b0b910bef27bc761"
format_stamp: "Formatted at 2026-09-26 16:36:09 on dist-test-slave-2zvq"
server_key: "be06c37c54b030634d2f79fe71c3abc1"
server_key_iv: "6bc0e2340a2460d826f9bf2e6e3824bb"
server_key_version: "encryptionkey@0"
I20260926 16:36:09.142593 29083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance:
uuid: "35edeb75b9ba4cb3b0b910bef27bc761"
format_stamp: "Formatted at 2026-09-26 16:36:09 on dist-test-slave-2zvq"
server_key: "be06c37c54b030634d2f79fe71c3abc1"
server_key_iv: "6bc0e2340a2460d826f9bf2e6e3824bb"
server_key_version: "encryptionkey@0"
I20260926 16:36:09.146358 29083 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260926 16:36:09.148793 29097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:09.149935 29083 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260926 16:36:09.150061 29083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-1/wal
uuid: "35edeb75b9ba4cb3b0b910bef27bc761"
format_stamp: "Formatted at 2026-09-26 16:36:09 on dist-test-slave-2zvq"
server_key: "be06c37c54b030634d2f79fe71c3abc1"
server_key_iv: "6bc0e2340a2460d826f9bf2e6e3824bb"
server_key_version: "encryptionkey@0"
I20260926 16:36:09.150180 29083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:09.163726 29083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:09.164395 29083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:09.164577 29083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:09.165150 29083 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:36:09.166061 29083 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:36:09.166106 29083 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:09.166170 29083 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:36:09.166205 29083 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:09.175221 29083 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.194:40305
I20260926 16:36:09.175246 29210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.194:40305 every 8 connection(s)
I20260926 16:36:09.176117 29083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
I20260926 16:36:09.183249 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 29083
I20260926 16:36:09.183383 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance
I20260926 16:36:09.183704 11299 external_mini_cluster.cc:1428] Setting key 942ce9567e9a1a49670553d45be981eb
I20260926 16:36:09.185633 28931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:09.177926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:34669 (local address 127.11.8.254:46715)
0926 16:36:09.178060 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:09.178064 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:09.179121 (+ 1057us) server_negotiation.cc:407] Connection header received
0926 16:36:09.180529 (+ 1408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:09.180533 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:09.180591 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:09.180695 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:09.181555 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:09.182090 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:09.183277 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:09.183614 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:09.183726 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:09.185072 (+ 1346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:09.185079 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:09.185085 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:36:09.185103 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:09.185130 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:36:09.185140 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:09.185228 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:09.185430 (+ 202us) server_negotiation.cc:299] Negotiation successful
0926 16:36:09.185495 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 16:36:09.186050 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.195:0
--local_ip_for_outbound_sockets=127.11.8.195
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:46715
--builtin_ntp_servers=127.11.8.212:38423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 16:36:09.186599 29213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:09.178350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46715 (local address 127.11.8.194:34669)
0926 16:36:09.178973 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:09.179011 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:09.180297 (+ 1286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:09.180865 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:09.180872 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:09.180954 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:36:09.181421 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:09.181432 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:09.182222 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:09.182229 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:09.183154 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:09.183163 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:09.183762 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:09.183766 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:09.183935 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:09.184712 (+ 777us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:36:09.184714 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:36:09.184767 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:36:09.184844 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:36:09.185239 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:36:09.185248 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:36:09.185254 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 16:36:09.185391 (+ 137us) client_negotiation.cc:240] Negotiation successful
0926 16:36:09.185633 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":101,"threads_started":1}
I20260926 16:36:09.188274 29211 heartbeater.cc:347] Connected to a master server at 127.11.8.254:46715
I20260926 16:36:09.188525 29211 heartbeater.cc:464] Registering TS with master...
I20260926 16:36:09.189121 29211 heartbeater.cc:511] Master 127.11.8.254:46715 requested a full tablet report, sending...
I20260926 16:36:09.190303 28892 ts_manager.cc:194] Registered new tserver with Master: 35edeb75b9ba4cb3b0b910bef27bc761 (127.11.8.194:40305)
I20260926 16:36:09.190889 28892 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.8.194:34669
W20260926 16:36:09.294926 29214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:09.295151 29214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:09.295186 29214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:09.299259 29214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:09.299309 29214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:09.299381 29214 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.195
I20260926 16:36:09.304291 29214 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:38423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:46715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29214
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.195
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:09.305308 29214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:09.306711 29214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:09.315968 29220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:09.315964 29222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:09.315963 29219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:09.316274 29214 server_base.cc:1034] running on GCE node
I20260926 16:36:09.316644 29214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:09.317180 29214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:09.318336 29214 hybrid_clock.cc:648] HybridClock initialized: now 1790440569318304 us; error 44 us; skew 500 ppm
I20260926 16:36:09.320108 29214 webserver.cc:460] Webserver started at http://127.11.8.195:36553/ using document root <none> and password file <none>
I20260926 16:36:09.320649 29214 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:09.320708 29214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:09.320941 29214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:09.322901 29214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-2/data/instance:
uuid: "075ae011e228467abf9b00f194fb4333"
format_stamp: "Formatted at 2026-09-26 16:36:09 on dist-test-slave-2zvq"
server_key: "525206fb72841c4a26e21e353ca8515b"
server_key_iv: "4224e686f0858335c194eeaa08acc7c6"
server_key_version: "encryptionkey@0"
I20260926 16:36:09.323421 29214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance:
uuid: "075ae011e228467abf9b00f194fb4333"
format_stamp: "Formatted at 2026-09-26 16:36:09 on dist-test-slave-2zvq"
server_key: "525206fb72841c4a26e21e353ca8515b"
server_key_iv: "4224e686f0858335c194eeaa08acc7c6"
server_key_version: "encryptionkey@0"
I20260926 16:36:09.326779 29214 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 16:36:09.328987 29228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:09.330099 29214 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:36:09.330219 29214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-2/wal
uuid: "075ae011e228467abf9b00f194fb4333"
format_stamp: "Formatted at 2026-09-26 16:36:09 on dist-test-slave-2zvq"
server_key: "525206fb72841c4a26e21e353ca8515b"
server_key_iv: "4224e686f0858335c194eeaa08acc7c6"
server_key_version: "encryptionkey@0"
I20260926 16:36:09.330365 29214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:09.343318 29214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:09.343904 29214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:09.344069 29214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:09.344570 29214 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:36:09.345386 29214 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:36:09.345463 29214 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:09.345525 29214 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:36:09.345570 29214 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:09.353665 29214 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.195:37811
I20260926 16:36:09.353731 29341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.195:37811 every 8 connection(s)
I20260926 16:36:09.354586 29214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
I20260926 16:36:09.362345 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 29214
I20260926 16:36:09.362473 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance
I20260926 16:36:09.362727 11299 external_mini_cluster.cc:1428] Setting key 78782cd158ae36600cc8341f16827b71
I20260926 16:36:09.363538 28931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:09.356261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:42761 (local address 127.11.8.254:46715)
0926 16:36:09.356401 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:09.356405 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:09.357350 (+ 945us) server_negotiation.cc:407] Connection header received
0926 16:36:09.358778 (+ 1428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:09.358781 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:09.358839 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:09.358945 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:09.359781 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:09.360308 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:09.361418 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:09.361637 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:09.361748 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:09.362924 (+ 1176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:09.362929 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:09.362931 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:36:09.362944 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:09.362968 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:36:09.362979 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:09.363065 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:09.363293 (+ 228us) server_negotiation.cc:299] Negotiation successful
0926 16:36:09.363395 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 16:36:09.364414 29344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:09.356621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:46715 (local address 127.11.8.195:42761)
0926 16:36:09.357215 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:09.357251 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:09.358581 (+ 1330us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:09.359105 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:09.359111 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:09.359205 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:36:09.359644 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:09.359656 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:09.360436 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:09.360440 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:09.361288 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:09.361296 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:09.361782 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:09.361786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:09.361947 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:09.362615 (+ 668us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:36:09.362617 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:36:09.362653 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:36:09.362709 (+ 56us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:36:09.363094 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:36:09.363102 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:36:09.363108 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 16:36:09.363236 (+ 128us) client_negotiation.cc:240] Negotiation successful
0926 16:36:09.363491 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":102,"threads_started":1}
I20260926 16:36:09.365546 29342 heartbeater.cc:347] Connected to a master server at 127.11.8.254:46715
I20260926 16:36:09.365846 29342 heartbeater.cc:464] Registering TS with master...
I20260926 16:36:09.366384 29342 heartbeater.cc:511] Master 127.11.8.254:46715 requested a full tablet report, sending...
I20260926 16:36:09.367280 28892 ts_manager.cc:194] Registered new tserver with Master: 075ae011e228467abf9b00f194fb4333 (127.11.8.195:37811)
I20260926 16:36:09.367831 28892 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.8.195:42761
I20260926 16:36:09.376578 11299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 16:36:09.382091 28931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:09.378472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52378 (local address 127.11.8.254:46715)
0926 16:36:09.378567 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:09.378570 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:09.378812 (+ 242us) server_negotiation.cc:407] Connection header received
0926 16:36:09.378908 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:09.378910 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:09.378947 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:09.379011 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:09.379386 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:09.379868 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:09.380724 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:09.380930 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:09.381008 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:09.381772 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:09.381775 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:09.381776 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:36:09.381786 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:09.381799 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:36:09.381805 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:09.381856 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:09.381951 (+ 95us) server_negotiation.cc:299] Negotiation successful
0926 16:36:09.381992 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260926 16:36:09.384683 28892 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52378:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:36:09.387562 28892 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:36:09.401373 29356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:09.396045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56552 (local address 127.11.8.194:40305)
0926 16:36:09.396533 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:09.396546 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:09.396574 (+ 28us) server_negotiation.cc:407] Connection header received
0926 16:36:09.396658 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:09.396664 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:09.396893 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:09.397050 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:36:09.397377 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:09.398597 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:09.399877 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:09.400681 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:09.400796 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:09.401003 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:09.401107 (+ 104us) server_negotiation.cc:299] Negotiation successful
0926 16:36:09.401212 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":112,"threads_started":1}
I20260926 16:36:09.401364 29353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:09.395307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:37929 (local address 127.0.0.1:41406)
0926 16:36:09.395615 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:09.395900 (+ 285us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:09.396018 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:09.397231 (+ 1213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:09.397234 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:09.397248 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:36:09.397343 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:09.397349 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:09.398788 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:09.398792 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:09.399948 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:09.399958 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:09.400945 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:09.400949 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:09.401043 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:09.401073 (+ 30us) client_negotiation.cc:769] Sending connection context
0926 16:36:09.401124 (+ 51us) client_negotiation.cc:240] Negotiation successful
0926 16:36:09.401227 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":59,"thread_start_us":64,"threads_started":1}
I20260926 16:36:09.401849 29357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:09.395477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:40305 (local address 127.0.0.1:56552)
0926 16:36:09.396426 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:09.396437 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:09.396502 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:09.397089 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:09.397095 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:09.397114 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:36:09.397235 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:09.397241 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:09.398730 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:09.398735 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:09.399743 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:09.399753 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:09.400807 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:09.400811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:09.400925 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:09.400967 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 16:36:09.401081 (+ 114us) client_negotiation.cc:240] Negotiation successful
0926 16:36:09.401713 (+ 632us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":893,"mutex_wait_us":53,"thread_start_us":102,"threads_started":1}
I20260926 16:36:09.401849 29354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:09.395458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41406 (local address 127.11.8.193:37929)
0926 16:36:09.396744 (+ 1286us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:09.396756 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:09.396779 (+ 23us) server_negotiation.cc:407] Connection header received
0926 16:36:09.396854 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:09.396859 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:09.397054 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:09.397237 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:36:09.397464 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:09.398682 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:09.400096 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:09.400817 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:09.401211 (+ 394us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:09.401394 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:09.401589 (+ 195us) server_negotiation.cc:299] Negotiation successful
0926 16:36:09.401690 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1152,"thread_start_us":107,"threads_started":1}
I20260926 16:36:09.402037 29358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:09.396451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40996 (local address 127.11.8.195:37811)
0926 16:36:09.396805 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:09.396816 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:09.396953 (+ 137us) server_negotiation.cc:407] Connection header received
0926 16:36:09.397062 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:09.397068 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:09.397228 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:09.397348 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:36:09.397635 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:09.398480 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:09.400483 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:09.401138 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:09.401226 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:09.401391 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:09.401785 (+ 394us) server_negotiation.cc:299] Negotiation successful
0926 16:36:09.401877 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":152,"threads_started":1}
I20260926 16:36:09.402310 29355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:09.396089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:37811 (local address 127.0.0.1:40996)
0926 16:36:09.396851 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:09.396865 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:09.396962 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:09.397417 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:09.397420 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:09.397436 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:36:09.397529 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:09.397533 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:09.398612 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:09.398616 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:09.400368 (+ 1752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:09.400376 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:09.401454 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:09.401460 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:09.401560 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:09.401592 (+ 32us) client_negotiation.cc:769] Sending connection context
0926 16:36:09.401657 (+ 65us) client_negotiation.cc:240] Negotiation successful
0926 16:36:09.401713 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":706,"mutex_wait_us":20,"thread_start_us":91,"threads_started":1}
I20260926 16:36:09.404729 29145 tablet_service.cc:1467] Processing CreateTablet for tablet 7f6e4d6284874520bba355754497fe51 (DEFAULT_TABLE table=test-table [id=caea2f67302c42b8b8e1680e03b930bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:36:09.404832 29276 tablet_service.cc:1467] Processing CreateTablet for tablet 7f6e4d6284874520bba355754497fe51 (DEFAULT_TABLE table=test-table [id=caea2f67302c42b8b8e1680e03b930bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:36:09.404832 29014 tablet_service.cc:1467] Processing CreateTablet for tablet 7f6e4d6284874520bba355754497fe51 (DEFAULT_TABLE table=test-table [id=caea2f67302c42b8b8e1680e03b930bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:36:09.405814 29276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f6e4d6284874520bba355754497fe51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:09.405814 29014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f6e4d6284874520bba355754497fe51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:09.405821 29145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f6e4d6284874520bba355754497fe51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:09.411605 29359 tablet_bootstrap.cc:492] T 7f6e4d6284874520bba355754497fe51 P 35edeb75b9ba4cb3b0b910bef27bc761: Bootstrap starting.
I20260926 16:36:09.413244 29361 tablet_bootstrap.cc:492] T 7f6e4d6284874520bba355754497fe51 P 4fb6e0d14b834c84ac8e6dd8efcc9934: Bootstrap starting.
I20260926 16:36:09.413808 29359 tablet_bootstrap.cc:654] T 7f6e4d6284874520bba355754497fe51 P 35edeb75b9ba4cb3b0b910bef27bc761: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:09.414569 29359 log.cc:824] T 7f6e4d6284874520bba355754497fe51 P 35edeb75b9ba4cb3b0b910bef27bc761: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:09.414585 29360 tablet_bootstrap.cc:492] T 7f6e4d6284874520bba355754497fe51 P 075ae011e228467abf9b00f194fb4333: Bootstrap starting.
I20260926 16:36:09.415153 29361 tablet_bootstrap.cc:654] T 7f6e4d6284874520bba355754497fe51 P 4fb6e0d14b834c84ac8e6dd8efcc9934: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:09.415938 29361 log.cc:824] T 7f6e4d6284874520bba355754497fe51 P 4fb6e0d14b834c84ac8e6dd8efcc9934: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:09.416550 29360 tablet_bootstrap.cc:654] T 7f6e4d6284874520bba355754497fe51 P 075ae011e228467abf9b00f194fb4333: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:09.416703 29359 tablet_bootstrap.cc:492] T 7f6e4d6284874520bba355754497fe51 P 35edeb75b9ba4cb3b0b910bef27bc761: No bootstrap required, opened a new log
I20260926 16:36:09.416898 29359 ts_tablet_manager.cc:1397] T 7f6e4d6284874520bba355754497fe51 P 35edeb75b9ba4cb3b0b910bef27bc761: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260926 16:36:09.417304 29360 log.cc:824] T 7f6e4d6284874520bba355754497fe51 P 075ae011e228467abf9b00f194fb4333: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:09.417901 29361 tablet_bootstrap.cc:492] T 7f6e4d6284874520bba355754497fe51 P 4fb6e0d14b834c84ac8e6dd8efcc9934: No bootstrap required, opened a new log
I20260926 16:36:09.418097 29361 ts_tablet_manager.cc:1397] T 7f6e4d6284874520bba355754497fe51 P 4fb6e0d14b834c84ac8e6dd8efcc9934: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260926 16:36:09.419279 29360 tablet_bootstrap.cc:492] T 7f6e4d6284874520bba355754497fe51 P 075ae011e228467abf9b00f194fb4333: No bootstrap required, opened a new log
I20260926 16:36:09.419454 29360 ts_tablet_manager.cc:1397] T 7f6e4d6284874520bba355754497fe51 P 075ae011e228467abf9b00f194fb4333: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260926 16:36:09.420509 29359 raft_consensus.cc:348] T 7f6e4d6284874520bba355754497fe51 P 35edeb75b9ba4cb3b0b910bef27bc761 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fb6e0d14b834c84ac8e6dd8efcc9934" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 37929 } } peers { permanent_uuid: "35edeb75b9ba4cb3b0b910bef27bc761" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 40305 } } peers { permanent_uuid: "075ae011e228467abf9b00f194fb4333" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37811 } }
I20260926 16:36:09.420768 29359 raft_consensus.cc:374] T 7f6e4d6284874520bba355754497fe51 P 35edeb75b9ba4cb3b0b910bef27bc761 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:36:09.420820 29359 raft_consensus.cc:729] T 7f6e4d6284874520bba355754497fe51 P 35edeb75b9ba4cb3b0b910bef27bc761 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35edeb75b9ba4cb3b0b910bef27bc761, State: Initialized, Role: FOLLOWER
I20260926 16:36:09.421247 29359 consensus_queue.cc:260] T 7f6e4d6284874520bba355754497fe51 P 35edeb75b9ba4cb3b0b910bef27bc761 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fb6e0d14b834c84ac8e6dd8efcc9934" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 37929 } } peers { permanent_uuid: "35edeb75b9ba4cb3b0b910bef27bc761" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 40305 } } peers { permanent_uuid: "075ae011e228467abf9b00f194fb4333" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37811 } }
I20260926 16:36:09.421898 29211 heartbeater.cc:503] Master 127.11.8.254:46715 was elected leader, sending a full tablet report...
I20260926 16:36:09.422233 29359 ts_tablet_manager.cc:1428] T 7f6e4d6284874520bba355754497fe51 P 35edeb75b9ba4cb3b0b910bef27bc761: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260926 16:36:09.422415 29361 raft_consensus.cc:348] T 7f6e4d6284874520bba355754497fe51 P 4fb6e0d14b834c84ac8e6dd8efcc9934 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fb6e0d14b834c84ac8e6dd8efcc9934" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 37929 } } peers { permanent_uuid: "35edeb75b9ba4cb3b0b910bef27bc761" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 40305 } } peers { permanent_uuid: "075ae011e228467abf9b00f194fb4333" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37811 } }
I20260926 16:36:09.422662 29361 raft_consensus.cc:374] T 7f6e4d6284874520bba355754497fe51 P 4fb6e0d14b834c84ac8e6dd8efcc9934 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:36:09.422714 29361 raft_consensus.cc:729] T 7f6e4d6284874520bba355754497fe51 P 4fb6e0d14b834c84ac8e6dd8efcc9934 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fb6e0d14b834c84ac8e6dd8efcc9934, State: Initialized, Role: FOLLOWER
I20260926 16:36:09.423116 29361 consensus_queue.cc:260] T 7f6e4d6284874520bba355754497fe51 P 4fb6e0d14b834c84ac8e6dd8efcc9934 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fb6e0d14b834c84ac8e6dd8efcc9934" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 37929 } } peers { permanent_uuid: "35edeb75b9ba4cb3b0b910bef27bc761" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 40305 } } peers { permanent_uuid: "075ae011e228467abf9b00f194fb4333" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37811 } }
I20260926 16:36:09.423434 29360 raft_consensus.cc:348] T 7f6e4d6284874520bba355754497fe51 P 075ae011e228467abf9b00f194fb4333 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fb6e0d14b834c84ac8e6dd8efcc9934" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 37929 } } peers { permanent_uuid: "35edeb75b9ba4cb3b0b910bef27bc761" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 40305 } } peers { permanent_uuid: "075ae011e228467abf9b00f194fb4333" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37811 } }
I20260926 16:36:09.423656 29360 raft_consensus.cc:374] T 7f6e4d6284874520bba355754497fe51 P 075ae011e228467abf9b00f194fb4333 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:36:09.423698 29360 raft_consensus.cc:729] T 7f6e4d6284874520bba355754497fe51 P 075ae011e228467abf9b00f194fb4333 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 075ae011e228467abf9b00f194fb4333, State: Initialized, Role: FOLLOWER
I20260926 16:36:09.423732 29080 heartbeater.cc:503] Master 127.11.8.254:46715 was elected leader, sending a full tablet report...
I20260926 16:36:09.424027 29361 ts_tablet_manager.cc:1428] T 7f6e4d6284874520bba355754497fe51 P 4fb6e0d14b834c84ac8e6dd8efcc9934: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260926 16:36:09.424090 29360 consensus_queue.cc:260] T 7f6e4d6284874520bba355754497fe51 P 075ae011e228467abf9b00f194fb4333 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fb6e0d14b834c84ac8e6dd8efcc9934" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 37929 } } peers { permanent_uuid: "35edeb75b9ba4cb3b0b910bef27bc761" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 40305 } } peers { permanent_uuid: "075ae011e228467abf9b00f194fb4333" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37811 } }
I20260926 16:36:09.424630 29342 heartbeater.cc:503] Master 127.11.8.254:46715 was elected leader, sending a full tablet report...
I20260926 16:36:09.426263 29360 ts_tablet_manager.cc:1428] T 7f6e4d6284874520bba355754497fe51 P 075ae011e228467abf9b00f194fb4333: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
W20260926 16:36:09.427014 29212 tablet.cc:2369] T 7f6e4d6284874520bba355754497fe51 P 35edeb75b9ba4cb3b0b910bef27bc761: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 16:36:09.498461 29081 tablet.cc:2369] T 7f6e4d6284874520bba355754497fe51 P 4fb6e0d14b834c84ac8e6dd8efcc9934: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:36:09.578035 29367 raft_consensus.cc:482] T 7f6e4d6284874520bba355754497fe51 P 075ae011e228467abf9b00f194fb4333 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:36:09.578266 29367 raft_consensus.cc:504] T 7f6e4d6284874520bba355754497fe51 P 075ae011e228467abf9b00f194fb4333 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fb6e0d14b834c84ac8e6dd8efcc9934" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 37929 } } peers { permanent_uuid: "35edeb75b9ba4cb3b0b910bef27bc761" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 40305 } } peers { permanent_uuid: "075ae011e228467abf9b00f194fb4333" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37811 } }
I20260926 16:36:09.579540 29367 leader_election.cc:288] T 7f6e4d6284874520bba355754497fe51 P 075ae011e228467abf9b00f194fb4333 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4fb6e0d14b834c84ac8e6dd8efcc9934 (127.11.8.193:37929), 35edeb75b9ba4cb3b0b910bef27bc761 (127.11.8.194:40305)
I20260926 16:36:09.583134 29354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:09.579869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:40745 (local address 127.11.8.193:37929)
0926 16:36:09.580036 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:09.580040 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:09.580060 (+ 20us) server_negotiation.cc:407] Connection header received
0926 16:36:09.580120 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:09.580126 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:09.580188 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:09.580301 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:36:09.580567 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:09.581069 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:09.582110 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:09.582584 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:09.582711 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:09.582762 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:09.582948 (+ 186us) server_negotiation.cc:299] Negotiation successful
0926 16:36:09.583013 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 16:36:09.583134 29344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:09.579742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:37929 (local address 127.11.8.195:40745)
0926 16:36:09.579884 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:09.579901 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:09.580001 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:09.580312 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:09.580315 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:09.580333 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:36:09.580450 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:09.580456 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:09.581187 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:09.581190 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:09.581984 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:09.581993 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:09.582705 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:09.582709 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:09.582822 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:09.582840 (+ 18us) client_negotiation.cc:769] Sending connection context
0926 16:36:09.582931 (+ 91us) client_negotiation.cc:240] Negotiation successful
0926 16:36:09.583002 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260926 16:36:09.583549 29356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:09.580059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:51689 (local address 127.11.8.194:40305)
0926 16:36:09.580183 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:09.580187 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:09.580327 (+ 140us) server_negotiation.cc:407] Connection header received
0926 16:36:09.580477 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:09.580480 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:09.580524 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:09.580608 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:36:09.580883 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:09.581366 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:09.582380 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:09.582750 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:09.582883 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:09.582929 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:09.583342 (+ 413us) server_negotiation.cc:299] Negotiation successful
0926 16:36:09.583429 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 16:36:09.583894 29368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:09.579877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:40305 (local address 127.11.8.195:51689)
0926 16:36:09.580215 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:09.580229 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:09.580354 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:09.580618 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:09.580621 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:09.580636 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:36:09.580761 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:09.580766 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:09.581476 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:09.581482 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:09.582231 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:09.582240 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:09.582874 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:09.582878 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:09.583198 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:09.583222 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 16:36:09.583687 (+ 465us) client_negotiation.cc:240] Negotiation successful
0926 16:36:09.583762 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":119,"threads_started":1}
I20260926 16:36:09.583978 29034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f6e4d6284874520bba355754497fe51" candidate_uuid: "075ae011e228467abf9b00f194fb4333" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fb6e0d14b834c84ac8e6dd8efcc9934" is_pre_election: true
I20260926 16:36:09.584348 29034 raft_consensus.cc:2454] T 7f6e4d6284874520bba355754497fe51 P 4fb6e0d14b834c84ac8e6dd8efcc9934 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 075ae011e228467abf9b00f194fb4333 in term 0.
I20260926 16:36:09.584604 29165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f6e4d6284874520bba355754497fe51" candidate_uuid: "075ae011e228467abf9b00f194fb4333" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35edeb75b9ba4cb3b0b910bef27bc761" is_pre_election: true
I20260926 16:36:09.584846 29231 leader_election.cc:302] T 7f6e4d6284874520bba355754497fe51 P 075ae011e228467abf9b00f194fb4333 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 075ae011e228467abf9b00f194fb4333, 4fb6e0d14b834c84ac8e6dd8efcc9934; no voters:
I20260926 16:36:09.584877 29165 raft_consensus.cc:2454] T 7f6e4d6284874520bba355754497fe51 P 35edeb75b9ba4cb3b0b910bef27bc761 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 075ae011e228467abf9b00f194fb4333 in term 0.
I20260926 16:36:09.585099 29367 raft_consensus.cc:2781] T 7f6e4d6284874520bba355754497fe51 P 075ae011e228467abf9b00f194fb4333 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:36:09.585187 29367 raft_consensus.cc:482] T 7f6e4d6284874520bba355754497fe51 P 075ae011e228467abf9b00f194fb4333 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:36:09.585243 29367 raft_consensus.cc:3037] T 7f6e4d6284874520bba355754497fe51 P 075ae011e228467abf9b00f194fb4333 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:09.586295 29367 raft_consensus.cc:504] T 7f6e4d6284874520bba355754497fe51 P 075ae011e228467abf9b00f194fb4333 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fb6e0d14b834c84ac8e6dd8efcc9934" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 37929 } } peers { permanent_uuid: "35edeb75b9ba4cb3b0b910bef27bc761" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 40305 } } peers { permanent_uuid: "075ae011e228467abf9b00f194fb4333" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37811 } }
I20260926 16:36:09.586622 29367 leader_election.cc:288] T 7f6e4d6284874520bba355754497fe51 P 075ae011e228467abf9b00f194fb4333 [CANDIDATE]: Term 1 election: Requested vote from peers 4fb6e0d14b834c84ac8e6dd8efcc9934 (127.11.8.193:37929), 35edeb75b9ba4cb3b0b910bef27bc761 (127.11.8.194:40305)
I20260926 16:36:09.586977 29034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f6e4d6284874520bba355754497fe51" candidate_uuid: "075ae011e228467abf9b00f194fb4333" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fb6e0d14b834c84ac8e6dd8efcc9934"
I20260926 16:36:09.587119 29034 raft_consensus.cc:3037] T 7f6e4d6284874520bba355754497fe51 P 4fb6e0d14b834c84ac8e6dd8efcc9934 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:09.587121 29165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f6e4d6284874520bba355754497fe51" candidate_uuid: "075ae011e228467abf9b00f194fb4333" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35edeb75b9ba4cb3b0b910bef27bc761"
I20260926 16:36:09.587251 29165 raft_consensus.cc:3037] T 7f6e4d6284874520bba355754497fe51 P 35edeb75b9ba4cb3b0b910bef27bc761 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:09.588135 29034 raft_consensus.cc:2454] T 7f6e4d6284874520bba355754497fe51 P 4fb6e0d14b834c84ac8e6dd8efcc9934 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 075ae011e228467abf9b00f194fb4333 in term 1.
I20260926 16:36:09.588343 29165 raft_consensus.cc:2454] T 7f6e4d6284874520bba355754497fe51 P 35edeb75b9ba4cb3b0b910bef27bc761 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 075ae011e228467abf9b00f194fb4333 in term 1.
I20260926 16:36:09.588449 29231 leader_election.cc:302] T 7f6e4d6284874520bba355754497fe51 P 075ae011e228467abf9b00f194fb4333 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 075ae011e228467abf9b00f194fb4333, 4fb6e0d14b834c84ac8e6dd8efcc9934; no voters:
I20260926 16:36:09.588635 29367 raft_consensus.cc:2781] T 7f6e4d6284874520bba355754497fe51 P 075ae011e228467abf9b00f194fb4333 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:36:09.588860 29367 raft_consensus.cc:686] T 7f6e4d6284874520bba355754497fe51 P 075ae011e228467abf9b00f194fb4333 [term 1 LEADER]: Becoming Leader. State: Replica: 075ae011e228467abf9b00f194fb4333, State: Running, Role: LEADER
I20260926 16:36:09.589190 29367 consensus_queue.cc:237] T 7f6e4d6284874520bba355754497fe51 P 075ae011e228467abf9b00f194fb4333 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fb6e0d14b834c84ac8e6dd8efcc9934" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 37929 } } peers { permanent_uuid: "35edeb75b9ba4cb3b0b910bef27bc761" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 40305 } } peers { permanent_uuid: "075ae011e228467abf9b00f194fb4333" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37811 } }
I20260926 16:36:09.592268 28890 catalog_manager.cc:5495] T 7f6e4d6284874520bba355754497fe51 P 075ae011e228467abf9b00f194fb4333 reported cstate change: term changed from 0 to 1, leader changed from <none> to 075ae011e228467abf9b00f194fb4333 (127.11.8.195). New cstate: current_term: 1 leader_uuid: "075ae011e228467abf9b00f194fb4333" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fb6e0d14b834c84ac8e6dd8efcc9934" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 37929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35edeb75b9ba4cb3b0b910bef27bc761" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 40305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "075ae011e228467abf9b00f194fb4333" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 37811 } health_report { overall_health: HEALTHY } } }
W20260926 16:36:09.605494 29343 tablet.cc:2369] T 7f6e4d6284874520bba355754497fe51 P 075ae011e228467abf9b00f194fb4333: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:36:09.638607 29358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:09.634777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41004 (local address 127.11.8.195:37811)
0926 16:36:09.634910 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:09.634915 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:09.634931 (+ 16us) server_negotiation.cc:407] Connection header received
0926 16:36:09.635060 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:09.635063 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:09.635123 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:09.635461 (+ 338us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:36:09.635724 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:09.636222 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:09.637125 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:09.637351 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:09.637456 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:09.637770 (+ 314us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:36:09.638179 (+ 409us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:36:09.638341 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:09.638390 (+ 49us) server_negotiation.cc:299] Negotiation successful
0926 16:36:09.638440 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 16:36:09.644447 29034 raft_consensus.cc:1261] T 7f6e4d6284874520bba355754497fe51 P 4fb6e0d14b834c84ac8e6dd8efcc9934 [term 1 FOLLOWER]: Refusing update from remote peer 075ae011e228467abf9b00f194fb4333: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:36:09.644618 29165 raft_consensus.cc:1261] T 7f6e4d6284874520bba355754497fe51 P 35edeb75b9ba4cb3b0b910bef27bc761 [term 1 FOLLOWER]: Refusing update from remote peer 075ae011e228467abf9b00f194fb4333: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:36:09.645190 29367 consensus_queue.cc:1035] T 7f6e4d6284874520bba355754497fe51 P 075ae011e228467abf9b00f194fb4333 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4fb6e0d14b834c84ac8e6dd8efcc9934" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 37929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:36:09.645418 29369 consensus_queue.cc:1035] T 7f6e4d6284874520bba355754497fe51 P 075ae011e228467abf9b00f194fb4333 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35edeb75b9ba4cb3b0b910bef27bc761" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 40305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:36:09.653211 29374 mvcc.cc:205] Tried to move back new op lower bound from 7333644573255815168 to 7333644573040046080. Current Snapshot: MvccSnapshot[applied={T|T < 7333644573255815168}]
I20260926 16:36:09.663046 28892 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:36:09.663192 28892 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:36:09.665512 28892 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 18f43254405d4605b6484c8be9a1ccf8: Sending DeleteTablet for 3 replicas of tablet 7f6e4d6284874520bba355754497fe51
I20260926 16:36:09.666306 29276 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f6e4d6284874520bba355754497fe51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:36:09 UTC) from {username='slave'} at 127.0.0.1:40996
I20260926 16:36:09.666307 29014 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f6e4d6284874520bba355754497fe51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:36:09 UTC) from {username='slave'} at 127.0.0.1:41406
I20260926 16:36:09.666786 29145 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f6e4d6284874520bba355754497fe51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:36:09 UTC) from {username='slave'} at 127.0.0.1:56552
I20260926 16:36:09.666882 29382 tablet_replica.cc:330] T 7f6e4d6284874520bba355754497fe51 P 4fb6e0d14b834c84ac8e6dd8efcc9934: stopping tablet replica
I20260926 16:36:09.667120 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 28952
I20260926 16:36:09.667207 29382 raft_consensus.cc:2229] T 7f6e4d6284874520bba355754497fe51 P 4fb6e0d14b834c84ac8e6dd8efcc9934 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:36:09.667707 29383 tablet_replica.cc:330] T 7f6e4d6284874520bba355754497fe51 P 075ae011e228467abf9b00f194fb4333: stopping tablet replica
I20260926 16:36:09.667989 29383 raft_consensus.cc:2229] T 7f6e4d6284874520bba355754497fe51 P 075ae011e228467abf9b00f194fb4333 [term 1 LEADER]: Raft consensus shutting down.
I20260926 16:36:09.668354 29383 raft_consensus.cc:2258] T 7f6e4d6284874520bba355754497fe51 P 075ae011e228467abf9b00f194fb4333 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:36:09.670548 29384 tablet_replica.cc:330] T 7f6e4d6284874520bba355754497fe51 P 35edeb75b9ba4cb3b0b910bef27bc761: stopping tablet replica
I20260926 16:36:09.670836 29384 raft_consensus.cc:2229] T 7f6e4d6284874520bba355754497fe51 P 35edeb75b9ba4cb3b0b910bef27bc761 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:36:09.671101 29384 raft_consensus.cc:2258] T 7f6e4d6284874520bba355754497fe51 P 35edeb75b9ba4cb3b0b910bef27bc761 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:36:09.671626 29383 ts_tablet_manager.cc:1905] T 7f6e4d6284874520bba355754497fe51 P 075ae011e228467abf9b00f194fb4333: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:36:09.672619 29384 ts_tablet_manager.cc:1905] T 7f6e4d6284874520bba355754497fe51 P 35edeb75b9ba4cb3b0b910bef27bc761: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:36:09.674650 29383 ts_tablet_manager.cc:1918] T 7f6e4d6284874520bba355754497fe51 P 075ae011e228467abf9b00f194fb4333: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:36:09.674741 29383 log.cc:1196] T 7f6e4d6284874520bba355754497fe51 P 075ae011e228467abf9b00f194fb4333: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-2/wal/wals/7f6e4d6284874520bba355754497fe51
I20260926 16:36:09.675064 29383 ts_tablet_manager.cc:1939] T 7f6e4d6284874520bba355754497fe51 P 075ae011e228467abf9b00f194fb4333: Deleting consensus metadata
W20260926 16:36:09.675133 28878 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.8.193:37929 (error 108)
I20260926 16:36:09.675884 29384 ts_tablet_manager.cc:1918] T 7f6e4d6284874520bba355754497fe51 P 35edeb75b9ba4cb3b0b910bef27bc761: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:36:09.675964 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 29083
I20260926 16:36:09.675977 29384 log.cc:1196] T 7f6e4d6284874520bba355754497fe51 P 35edeb75b9ba4cb3b0b910bef27bc761: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790440324823152-11299-0/minicluster-data/ts-1/wal/wals/7f6e4d6284874520bba355754497fe51
I20260926 16:36:09.676265 29384 ts_tablet_manager.cc:1939] T 7f6e4d6284874520bba355754497fe51 P 35edeb75b9ba4cb3b0b910bef27bc761: Deleting consensus metadata
I20260926 16:36:09.676693 28877 catalog_manager.cc:4841] TS 075ae011e228467abf9b00f194fb4333 (127.11.8.195:37811): tablet 7f6e4d6284874520bba355754497fe51 (table test-table [id=caea2f67302c42b8b8e1680e03b930bb]) successfully deleted
I20260926 16:36:09.682986 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 29214
I20260926 16:36:09.683205 29355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:09.682874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:37929 (local address 127.0.0.1:41422)
0926 16:36:09.683043 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:09.683110 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.193:37929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260926 16:36:09.683432 29355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:09.683120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:40305 (local address 127.0.0.1:56568)
0926 16:36:09.683342 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:09.683387 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.194:40305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172}
W20260926 16:36:09.683854 28876 catalog_manager.cc:4584] TS 35edeb75b9ba4cb3b0b910bef27bc761 (127.11.8.194:40305): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7f6e4d6284874520bba355754497fe51: Network error: Client connection negotiation failed: client connection to 127.11.8.194:40305: connect: Connection refused (error 111)
I20260926 16:36:09.687860 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 28861
2026-09-26T16:36:09Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1164 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:36:09 dist-test-slave-2zvq krb5kdc[29388](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:36:09 dist-test-slave-2zvq krb5kdc[29388](info): set up 2 sockets
Sep 26 16:36:09 dist-test-slave-2zvq krb5kdc[29388](info): commencing operation
krb5kdc: starting...
W20260926 16:36:10.739934 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:36:10 dist-test-slave-2zvq krb5kdc[29388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:36:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:36:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
I20260926 16:36:10.860865 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:43631
--webserver_interface=127.11.8.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:42439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:43631
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:36:10.972270 29401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:10.972508 29401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:10.972546 29401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:10.976467 29401 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:36:10.976516 29401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:10.976534 29401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:10.976550 29401 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:36:10.976608 29401 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:36:10.981935 29401 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:42439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:43631
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:43631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:10.982930 29401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:10.983934 29401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:10.992148 29407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:10.992174 29406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:10.992328 29401 server_base.cc:1034] running on GCE node
W20260926 16:36:10.992177 29409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:10.992942 29401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:10.993996 29401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:10.995187 29401 hybrid_clock.cc:648] HybridClock initialized: now 1790440570995155 us; error 52 us; skew 500 ppm
Sep 26 16:36:10 dist-test-slave-2zvq krb5kdc[29388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440570, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:36:10.997903 29401 init.cc:377] Logged in from keytab as kudu/127.11.8.254@KRBTEST.COM (short username kudu)
I20260926 16:36:10.998874 29401 webserver.cc:460] Webserver started at http://127.11.8.254:37901/ using document root <none> and password file <none>
I20260926 16:36:10.999430 29401 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:10.999521 29401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:10.999751 29401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:11.001681 29401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "b007db5545174d659f7fef0fb24486df"
format_stamp: "Formatted at 2026-09-26 16:36:10 on dist-test-slave-2zvq"
server_key: "fe67c0cb60797da33216b1f39d98dcee"
server_key_iv: "ba2ff0e5eea99fbae032ec49a90d4daf"
server_key_version: "encryptionkey@0"
I20260926 16:36:11.002230 29401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "b007db5545174d659f7fef0fb24486df"
format_stamp: "Formatted at 2026-09-26 16:36:10 on dist-test-slave-2zvq"
server_key: "fe67c0cb60797da33216b1f39d98dcee"
server_key_iv: "ba2ff0e5eea99fbae032ec49a90d4daf"
server_key_version: "encryptionkey@0"
I20260926 16:36:11.005811 29401 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:36:11.008299 29416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:11.009528 29401 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260926 16:36:11.009676 29401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "b007db5545174d659f7fef0fb24486df"
format_stamp: "Formatted at 2026-09-26 16:36:10 on dist-test-slave-2zvq"
server_key: "fe67c0cb60797da33216b1f39d98dcee"
server_key_iv: "ba2ff0e5eea99fbae032ec49a90d4daf"
server_key_version: "encryptionkey@0"
I20260926 16:36:11.009793 29401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:11.022464 29401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:11.025135 29401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:11.025389 29401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:11.032687 29401 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:43631
I20260926 16:36:11.032694 29468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:43631 every 8 connection(s)
I20260926 16:36:11.033603 29401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:36:11.036377 29469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:11.036867 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 29401
I20260926 16:36:11.037019 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:36:11.037287 11299 external_mini_cluster.cc:1428] Setting key d44deae14a535789183c9bd9b7b2f6c4
I20260926 16:36:11.042999 29469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b007db5545174d659f7fef0fb24486df: Bootstrap starting.
I20260926 16:36:11.045559 29469 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b007db5545174d659f7fef0fb24486df: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:11.046368 29469 log.cc:824] T 00000000000000000000000000000000 P b007db5545174d659f7fef0fb24486df: Log is configured to *not* fsync() on all Append() calls
Sep 26 16:36:11 dist-test-slave-2zvq krb5kdc[29388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:36:11.048623 29469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b007db5545174d659f7fef0fb24486df: No bootstrap required, opened a new log
I20260926 16:36:11.052042 29472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:11.038689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36542 (local address 127.11.8.254:43631)
0926 16:36:11.039092 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:11.039111 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:11.039154 (+ 43us) server_negotiation.cc:407] Connection header received
0926 16:36:11.039986 (+ 832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:11.040009 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:11.040428 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:11.040777 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:11.041441 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:11.042929 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:11.043849 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:11.044553 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:11.044709 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:11.047542 (+ 2833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:11.047547 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:11.047560 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:11.047586 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:11.049268 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:11.049748 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:11.049752 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:11.049757 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:11.049859 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:11.050136 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:11.050139 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:11.050140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:11.050557 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:11.050696 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:11.051043 (+ 347us) server_negotiation.cc:299] Negotiation successful
0926 16:36:11.051336 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":102,"threads_started":1}
I20260926 16:36:11.052671 29469 raft_consensus.cc:348] T 00000000000000000000000000000000 P b007db5545174d659f7fef0fb24486df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b007db5545174d659f7fef0fb24486df" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 43631 } }
I20260926 16:36:11.052914 29469 raft_consensus.cc:374] T 00000000000000000000000000000000 P b007db5545174d659f7fef0fb24486df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:36:11.053002 29469 raft_consensus.cc:729] T 00000000000000000000000000000000 P b007db5545174d659f7fef0fb24486df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b007db5545174d659f7fef0fb24486df, State: Initialized, Role: FOLLOWER
I20260926 16:36:11.053422 29469 consensus_queue.cc:260] T 00000000000000000000000000000000 P b007db5545174d659f7fef0fb24486df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b007db5545174d659f7fef0fb24486df" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 43631 } }
I20260926 16:36:11.053570 29469 raft_consensus.cc:388] T 00000000000000000000000000000000 P b007db5545174d659f7fef0fb24486df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:36:11.053651 29469 raft_consensus.cc:482] T 00000000000000000000000000000000 P b007db5545174d659f7fef0fb24486df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:36:11.053766 29469 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b007db5545174d659f7fef0fb24486df [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:11.054800 29469 raft_consensus.cc:504] T 00000000000000000000000000000000 P b007db5545174d659f7fef0fb24486df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b007db5545174d659f7fef0fb24486df" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 43631 } }
I20260926 16:36:11.055130 29469 leader_election.cc:302] T 00000000000000000000000000000000 P b007db5545174d659f7fef0fb24486df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b007db5545174d659f7fef0fb24486df; no voters:
I20260926 16:36:11.055424 29469 leader_election.cc:288] T 00000000000000000000000000000000 P b007db5545174d659f7fef0fb24486df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:36:11.055569 29474 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b007db5545174d659f7fef0fb24486df [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:36:11.055820 29474 raft_consensus.cc:686] T 00000000000000000000000000000000 P b007db5545174d659f7fef0fb24486df [term 1 LEADER]: Becoming Leader. State: Replica: b007db5545174d659f7fef0fb24486df, State: Running, Role: LEADER
I20260926 16:36:11.056111 29474 consensus_queue.cc:237] T 00000000000000000000000000000000 P b007db5545174d659f7fef0fb24486df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b007db5545174d659f7fef0fb24486df" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 43631 } }
I20260926 16:36:11.056631 29469 sys_catalog.cc:564] T 00000000000000000000000000000000 P b007db5545174d659f7fef0fb24486df [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:36:11.058003 29476 sys_catalog.cc:455] T 00000000000000000000000000000000 P b007db5545174d659f7fef0fb24486df [sys.catalog]: SysCatalogTable state changed. Reason: New leader b007db5545174d659f7fef0fb24486df. Latest consensus state: current_term: 1 leader_uuid: "b007db5545174d659f7fef0fb24486df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b007db5545174d659f7fef0fb24486df" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 43631 } } }
I20260926 16:36:11.058130 29476 sys_catalog.cc:458] T 00000000000000000000000000000000 P b007db5545174d659f7fef0fb24486df [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:11.058508 29484 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:36:11.058549 29475 sys_catalog.cc:455] T 00000000000000000000000000000000 P b007db5545174d659f7fef0fb24486df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b007db5545174d659f7fef0fb24486df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b007db5545174d659f7fef0fb24486df" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 43631 } } }
I20260926 16:36:11.058740 29475 sys_catalog.cc:458] T 00000000000000000000000000000000 P b007db5545174d659f7fef0fb24486df [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:11.061117 29484 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:36:11.066107 29484 catalog_manager.cc:1347] Generated new cluster ID: 28745102ba204c5c94bd0e05a54ac258
I20260926 16:36:11.066167 29484 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:36:11.092875 29484 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:36:11.093971 29484 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:36:11.101992 29484 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b007db5545174d659f7fef0fb24486df: Generated new TSK 0
I20260926 16:36:11.102826 29484 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.8.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab.
Entry for principal kudu/127.11.8.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab.
I20260926 16:36:11.130153 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.193:0
--local_ip_for_outbound_sockets=127.11.8.193
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:43631
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:42439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:36:11.240875 29495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:11.241133 29495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:11.241209 29495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:11.245118 29495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:11.245209 29495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:11.245298 29495 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.193
I20260926 16:36:11.250695 29495 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:42439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:43631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.193
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:11.251540 29495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:11.252408 29495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:11.261654 29503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:11.261684 29500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:11.261698 29495 server_base.cc:1034] running on GCE node
W20260926 16:36:11.261660 29501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:11.262202 29495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:11.262842 29495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:11.264024 29495 hybrid_clock.cc:648] HybridClock initialized: now 1790440571263990 us; error 45 us; skew 500 ppm
Sep 26 16:36:11 dist-test-slave-2zvq krb5kdc[29388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440571, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:36:11.266562 29495 init.cc:377] Logged in from keytab as kudu/127.11.8.193@KRBTEST.COM (short username kudu)
I20260926 16:36:11.267318 29495 webserver.cc:460] Webserver started at http://127.11.8.193:45217/ using document root <none> and password file <none>
I20260926 16:36:11.267841 29495 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:11.267931 29495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:11.268151 29495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:11.270092 29495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-0/data/instance:
uuid: "d3776302e9614e7c8a50bfb0ec5684af"
format_stamp: "Formatted at 2026-09-26 16:36:11 on dist-test-slave-2zvq"
server_key: "eaf4414edaa83cb67fbc6dea5fa9c87c"
server_key_iv: "11243db1630eb238127c67e10356f327"
server_key_version: "encryptionkey@0"
I20260926 16:36:11.270694 29495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance:
uuid: "d3776302e9614e7c8a50bfb0ec5684af"
format_stamp: "Formatted at 2026-09-26 16:36:11 on dist-test-slave-2zvq"
server_key: "eaf4414edaa83cb67fbc6dea5fa9c87c"
server_key_iv: "11243db1630eb238127c67e10356f327"
server_key_version: "encryptionkey@0"
I20260926 16:36:11.274206 29495 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:36:11.276485 29510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:11.277760 29495 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260926 16:36:11.277848 29495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-0/wal
uuid: "d3776302e9614e7c8a50bfb0ec5684af"
format_stamp: "Formatted at 2026-09-26 16:36:11 on dist-test-slave-2zvq"
server_key: "eaf4414edaa83cb67fbc6dea5fa9c87c"
server_key_iv: "11243db1630eb238127c67e10356f327"
server_key_version: "encryptionkey@0"
I20260926 16:36:11.277933 29495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:11.287334 29495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:11.289849 29495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:11.289999 29495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:11.290534 29495 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:36:11.291240 29495 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:36:11.291285 29495 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:11.291320 29495 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:36:11.291342 29495 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:11.300083 29495 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.193:42845
I20260926 16:36:11.300105 29623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.193:42845 every 8 connection(s)
I20260926 16:36:11.300915 29495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
I20260926 16:36:11.306061 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 29495
I20260926 16:36:11.306200 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance
I20260926 16:36:11.306550 11299 external_mini_cluster.cc:1428] Setting key c0de6b64f082169c559647c07583e256
WARNING: no policy specified for kudu/127.11.8.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.194@KRBTEST.COM" created.
Sep 26 16:36:11 dist-test-slave-2zvq krb5kdc[29388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440571, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:36:11.324895 29472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:11.302762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:43119 (local address 127.11.8.254:43631)
0926 16:36:11.302920 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:11.302925 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:11.303826 (+ 901us) server_negotiation.cc:407] Connection header received
0926 16:36:11.305114 (+ 1288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:11.305117 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:11.305163 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:11.305235 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:11.306564 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:11.307275 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:11.308650 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:11.309069 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:11.309187 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:11.320939 (+ 11752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:11.320942 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:11.320944 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:11.320970 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:11.322749 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:11.323275 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:11.323281 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:11.323286 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:11.323382 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:11.323724 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:11.323729 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:11.323730 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:11.323980 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:11.324088 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:11.324563 (+ 475us) server_negotiation.cc:299] Negotiation successful
0926 16:36:11.324698 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 16:36:11.325661 29626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:11.303157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:43631 (local address 127.11.8.193:43119)
0926 16:36:11.303668 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:11.303712 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:11.304901 (+ 1189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:11.305409 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:11.305415 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:11.305746 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:36:11.306420 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:11.306437 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:11.307417 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:11.307421 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:11.308527 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:11.308537 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:11.309251 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:11.309258 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:11.309476 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:11.318841 (+ 9365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:36:11.318862 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:36:11.320711 (+ 1849us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:36:11.322891 (+ 2180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:11.322896 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:11.322909 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:11.323172 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:11.323483 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:11.323485 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:11.323486 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:11.323623 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:11.324084 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:36:11.324089 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:36:11.324328 (+ 239us) client_negotiation.cc:769] Sending connection context
0926 16:36:11.324483 (+ 155us) client_negotiation.cc:240] Negotiation successful
0926 16:36:11.324739 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":110,"threads_started":1}
I20260926 16:36:11.327003 29624 heartbeater.cc:347] Connected to a master server at 127.11.8.254:43631
I20260926 16:36:11.327273 29624 heartbeater.cc:464] Registering TS with master...
I20260926 16:36:11.327865 29624 heartbeater.cc:511] Master 127.11.8.254:43631 requested a full tablet report, sending...
I20260926 16:36:11.329479 29433 ts_manager.cc:194] Registered new tserver with Master: d3776302e9614e7c8a50bfb0ec5684af (127.11.8.193:42845)
I20260926 16:36:11.330776 29433 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.193@KRBTEST.COM'} at 127.11.8.193:43119
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab.
Entry for principal kudu/127.11.8.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab.
I20260926 16:36:11.332882 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.194:0
--local_ip_for_outbound_sockets=127.11.8.194
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:43631
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:42439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:36:11.442607 29629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:11.442901 29629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:11.442981 29629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:11.447573 29629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:11.447659 29629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:11.447782 29629 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.194
I20260926 16:36:11.453595 29629 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:42439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:43631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29629
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.194
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:11.454777 29629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:11.455847 29629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:11.466207 29635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:11.466333 29634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:11.466441 29629 server_base.cc:1034] running on GCE node
W20260926 16:36:11.466643 29637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:11.467053 29629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:11.467609 29629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:11.468770 29629 hybrid_clock.cc:648] HybridClock initialized: now 1790440571468736 us; error 43 us; skew 500 ppm
Sep 26 16:36:11 dist-test-slave-2zvq krb5kdc[29388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440571, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:36:11.471403 29629 init.cc:377] Logged in from keytab as kudu/127.11.8.194@KRBTEST.COM (short username kudu)
I20260926 16:36:11.472177 29629 webserver.cc:460] Webserver started at http://127.11.8.194:39321/ using document root <none> and password file <none>
I20260926 16:36:11.472684 29629 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:11.472740 29629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:11.472913 29629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:11.474936 29629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-1/data/instance:
uuid: "3f339b8f27424a95b1f16a3fe71a25e0"
format_stamp: "Formatted at 2026-09-26 16:36:11 on dist-test-slave-2zvq"
server_key: "940c28bfe6579562d60d09292e46afac"
server_key_iv: "413a32008057d5d1e408eeaaabc5ec83"
server_key_version: "encryptionkey@0"
I20260926 16:36:11.475423 29629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance:
uuid: "3f339b8f27424a95b1f16a3fe71a25e0"
format_stamp: "Formatted at 2026-09-26 16:36:11 on dist-test-slave-2zvq"
server_key: "940c28bfe6579562d60d09292e46afac"
server_key_iv: "413a32008057d5d1e408eeaaabc5ec83"
server_key_version: "encryptionkey@0"
I20260926 16:36:11.479010 29629 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 16:36:11.481415 29644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:11.482745 29629 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:36:11.482890 29629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-1/wal
uuid: "3f339b8f27424a95b1f16a3fe71a25e0"
format_stamp: "Formatted at 2026-09-26 16:36:11 on dist-test-slave-2zvq"
server_key: "940c28bfe6579562d60d09292e46afac"
server_key_iv: "413a32008057d5d1e408eeaaabc5ec83"
server_key_version: "encryptionkey@0"
I20260926 16:36:11.483001 29629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:11.505167 29629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:11.508688 29629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:11.508888 29629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:11.509498 29629 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:36:11.510389 29629 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:36:11.510454 29629 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:11.510519 29629 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:36:11.510569 29629 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:11.519542 29629 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.194:41227
I20260926 16:36:11.519563 29757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.194:41227 every 8 connection(s)
I20260926 16:36:11.520431 29629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
I20260926 16:36:11.528955 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 29629
I20260926 16:36:11.529099 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance
I20260926 16:36:11.529419 11299 external_mini_cluster.cc:1428] Setting key be260295cc7dbf48fc272303046c8586
Sep 26 16:36:11 dist-test-slave-2zvq krb5kdc[29388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440571, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:36:11.533756 29472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:11.522064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:33853 (local address 127.11.8.254:43631)
0926 16:36:11.522206 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:11.522210 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:11.523066 (+ 856us) server_negotiation.cc:407] Connection header received
0926 16:36:11.524391 (+ 1325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:11.524394 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:11.524449 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:11.524553 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:11.525657 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:11.526183 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:11.527424 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:11.527654 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:11.527764 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:11.530334 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:11.530338 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:11.530340 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:11.530373 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:11.531789 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:11.532298 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:11.532303 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:11.532305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:11.532386 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:11.532714 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:11.532718 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:11.532720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:11.532900 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:11.533006 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:11.533487 (+ 481us) server_negotiation.cc:299] Negotiation successful
0926 16:36:11.533601 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 16:36:11.534651 29760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:11.522497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:43631 (local address 127.11.8.194:33853)
0926 16:36:11.522924 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:11.522965 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:11.524168 (+ 1203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:11.524721 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:11.524727 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:11.525058 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:36:11.525517 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:11.525530 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:11.526322 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:11.526332 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:11.527290 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:11.527300 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:11.527800 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:11.527804 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:11.527971 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:11.528544 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:36:11.528562 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:36:11.530132 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:36:11.531919 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:11.531925 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:11.531939 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:11.532192 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:11.532477 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:11.532481 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:11.532483 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:11.532619 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:11.533005 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:36:11.533013 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:36:11.533248 (+ 235us) client_negotiation.cc:769] Sending connection context
0926 16:36:11.533445 (+ 197us) client_negotiation.cc:240] Negotiation successful
0926 16:36:11.533750 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":117,"threads_started":1}
I20260926 16:36:11.535758 29758 heartbeater.cc:347] Connected to a master server at 127.11.8.254:43631
I20260926 16:36:11.536022 29758 heartbeater.cc:464] Registering TS with master...
I20260926 16:36:11.536662 29758 heartbeater.cc:511] Master 127.11.8.254:43631 requested a full tablet report, sending...
I20260926 16:36:11.537868 29433 ts_manager.cc:194] Registered new tserver with Master: 3f339b8f27424a95b1f16a3fe71a25e0 (127.11.8.194:41227)
I20260926 16:36:11.538545 29433 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.194@KRBTEST.COM'} at 127.11.8.194:33853
WARNING: no policy specified for kudu/127.11.8.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab.
Entry for principal kudu/127.11.8.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab.
I20260926 16:36:11.555281 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.195:0
--local_ip_for_outbound_sockets=127.11.8.195
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:43631
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:42439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:36:11.664721 29763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:11.664958 29763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:11.665000 29763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:11.668906 29763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:11.668953 29763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:11.669008 29763 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.195
I20260926 16:36:11.674013 29763 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:42439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:43631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.195
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:11.674914 29763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:11.675827 29763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 16:36:11.685371 29763 server_base.cc:1034] running on GCE node
W20260926 16:36:11.685381 29771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:11.685457 29768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:11.685388 29769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:11.686085 29763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:11.686841 29763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:11.688028 29763 hybrid_clock.cc:648] HybridClock initialized: now 1790440571688012 us; error 30 us; skew 500 ppm
Sep 26 16:36:11 dist-test-slave-2zvq krb5kdc[29388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440571, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:36:11.690771 29763 init.cc:377] Logged in from keytab as kudu/127.11.8.195@KRBTEST.COM (short username kudu)
I20260926 16:36:11.691699 29763 webserver.cc:460] Webserver started at http://127.11.8.195:45997/ using document root <none> and password file <none>
I20260926 16:36:11.692255 29763 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:11.692345 29763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:11.692576 29763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:11.694617 29763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-2/data/instance:
uuid: "f85a53e3e81748aeb5af3bd28c01516d"
format_stamp: "Formatted at 2026-09-26 16:36:11 on dist-test-slave-2zvq"
server_key: "f2897edcf4c3dbb57f9b80ccf5727df9"
server_key_iv: "4ab19bfad99fb2eccd99271cac14efe9"
server_key_version: "encryptionkey@0"
I20260926 16:36:11.695161 29763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance:
uuid: "f85a53e3e81748aeb5af3bd28c01516d"
format_stamp: "Formatted at 2026-09-26 16:36:11 on dist-test-slave-2zvq"
server_key: "f2897edcf4c3dbb57f9b80ccf5727df9"
server_key_iv: "4ab19bfad99fb2eccd99271cac14efe9"
server_key_version: "encryptionkey@0"
I20260926 16:36:11.698619 29763 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:36:11.700954 29778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:11.702096 29763 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:36:11.702222 29763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-2/wal
uuid: "f85a53e3e81748aeb5af3bd28c01516d"
format_stamp: "Formatted at 2026-09-26 16:36:11 on dist-test-slave-2zvq"
server_key: "f2897edcf4c3dbb57f9b80ccf5727df9"
server_key_iv: "4ab19bfad99fb2eccd99271cac14efe9"
server_key_version: "encryptionkey@0"
I20260926 16:36:11.702384 29763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:11.714627 29763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:11.717388 29763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:11.717584 29763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:11.718143 29763 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:36:11.718992 29763 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:36:11.719061 29763 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:11.719129 29763 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:36:11.719177 29763 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:11.727823 29763 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.195:40441
I20260926 16:36:11.727871 29891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.195:40441 every 8 connection(s)
I20260926 16:36:11.728709 29763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
I20260926 16:36:11.731748 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 29763
I20260926 16:36:11.731894 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance
I20260926 16:36:11.732273 11299 external_mini_cluster.cc:1428] Setting key d8a354f6dee9f19f55b1aae6df5857d3
Sep 26 16:36:11 dist-test-slave-2zvq krb5kdc[29388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440571, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:36:11.742007 29472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:11.730524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:45281 (local address 127.11.8.254:43631)
0926 16:36:11.730668 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:11.730672 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:11.731935 (+ 1263us) server_negotiation.cc:407] Connection header received
0926 16:36:11.733214 (+ 1279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:11.733218 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:11.733275 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:11.733353 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:11.734789 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:11.735307 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:11.736367 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:11.736592 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:11.736666 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:11.739091 (+ 2425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:11.739094 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:11.739096 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:11.739119 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:11.740341 (+ 1222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:11.740787 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:11.740789 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:11.740790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:11.740861 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:11.741139 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:11.741141 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:11.741142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:11.741296 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:11.741365 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:11.741797 (+ 432us) server_negotiation.cc:299] Negotiation successful
0926 16:36:11.741889 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 16:36:11.742830 29894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:11.730907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:43631 (local address 127.11.8.195:45281)
0926 16:36:11.731357 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:11.731402 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:11.732975 (+ 1573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:11.733565 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:11.733572 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:11.734017 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:36:11.734661 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:11.734677 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:11.735408 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:11.735410 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:11.736265 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:11.736274 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:11.736712 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:11.736714 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:11.736860 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:11.737460 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:36:11.737475 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:36:11.738946 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:36:11.740453 (+ 1507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:11.740458 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:11.740469 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:11.740700 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:11.740934 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:11.740936 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:11.740937 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:11.741059 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:11.741382 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:36:11.741387 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:36:11.741598 (+ 211us) client_negotiation.cc:769] Sending connection context
0926 16:36:11.741739 (+ 141us) client_negotiation.cc:240] Negotiation successful
0926 16:36:11.741974 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":111,"threads_started":1}
I20260926 16:36:11.743945 29892 heartbeater.cc:347] Connected to a master server at 127.11.8.254:43631
I20260926 16:36:11.744174 29892 heartbeater.cc:464] Registering TS with master...
I20260926 16:36:11.744676 29892 heartbeater.cc:511] Master 127.11.8.254:43631 requested a full tablet report, sending...
I20260926 16:36:11.745764 29433 ts_manager.cc:194] Registered new tserver with Master: f85a53e3e81748aeb5af3bd28c01516d (127.11.8.195:40441)
I20260926 16:36:11.746232 11299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 16:36:11.746448 29433 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.195@KRBTEST.COM'} at 127.11.8.195:45281
I20260926 16:36:11.755412 29472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:11.748399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36550 (local address 127.11.8.254:43631)
0926 16:36:11.748528 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:11.748533 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:11.748719 (+ 186us) server_negotiation.cc:407] Connection header received
0926 16:36:11.748793 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:11.748795 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:11.748829 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:11.748898 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:11.749529 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:11.750005 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:11.750886 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:11.751086 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:11.751161 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:11.752523 (+ 1362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:11.752525 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:11.752528 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:11.752549 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:11.753939 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:11.754418 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:11.754420 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:11.754421 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:11.754488 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:11.754740 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:11.754742 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:11.754743 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:11.754917 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:11.755024 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:11.755215 (+ 191us) server_negotiation.cc:299] Negotiation successful
0926 16:36:11.755282 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 16:36:11.758189 29433 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36550:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:36:11.761134 29433 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:36:11.775301 29905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:11.769598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:40441 (local address 127.0.0.1:51204)
0926 16:36:11.769867 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:11.769882 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:11.769966 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:11.770968 (+ 1002us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:11.770974 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:11.770995 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:36:11.771268 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:11.771277 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:11.772317 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:11.772321 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:11.773616 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:11.773626 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:11.774757 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:11.774764 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:11.774873 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:11.774910 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 16:36:11.775017 (+ 107us) client_negotiation.cc:240] Negotiation successful
0926 16:36:11.775131 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":3,"thread_start_us":113,"threads_started":1}
I20260926 16:36:11.775442 29908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:11.770102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51204 (local address 127.11.8.195:40441)
0926 16:36:11.770478 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:11.770490 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:11.770513 (+ 23us) server_negotiation.cc:407] Connection header received
0926 16:36:11.770591 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:11.770600 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:11.770809 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:11.770968 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:36:11.771405 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:11.772187 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:11.773770 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:11.774633 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:11.774746 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:11.774890 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:11.775206 (+ 316us) server_negotiation.cc:299] Negotiation successful
0926 16:36:11.775296 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":131,"threads_started":1}
I20260926 16:36:11.775607 29904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:11.769434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51218 (local address 127.11.8.193:42845)
0926 16:36:11.770844 (+ 1410us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:11.770858 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:11.770881 (+ 23us) server_negotiation.cc:407] Connection header received
0926 16:36:11.770974 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:11.770978 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:11.771140 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:11.771280 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:36:11.771484 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:11.772596 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:11.774082 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:11.774739 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:11.774840 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:11.774998 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:11.775374 (+ 376us) server_negotiation.cc:299] Negotiation successful
0926 16:36:11.775485 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":785,"thread_start_us":155,"threads_started":1}
I20260926 16:36:11.775607 29907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:11.770120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57330 (local address 127.11.8.194:41227)
0926 16:36:11.770591 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:11.770602 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:11.770627 (+ 25us) server_negotiation.cc:407] Connection header received
0926 16:36:11.770703 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:11.770708 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:11.770952 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:11.771099 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:36:11.771419 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:11.772596 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:11.774115 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:11.774909 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:11.775003 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:11.775122 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:11.775367 (+ 245us) server_negotiation.cc:299] Negotiation successful
0926 16:36:11.775451 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":111,"threads_started":1}
I20260926 16:36:11.776012 29906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:11.769973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:41227 (local address 127.0.0.1:57330)
0926 16:36:11.770510 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:11.770523 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:11.770605 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:11.771142 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:11.771145 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:11.771163 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:36:11.771276 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:11.771283 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:11.772775 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:11.772778 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:11.773982 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:11.773992 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:11.775015 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:11.775018 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:11.775100 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:11.775121 (+ 21us) client_negotiation.cc:769] Sending connection context
0926 16:36:11.775192 (+ 71us) client_negotiation.cc:240] Negotiation successful
0926 16:36:11.775258 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"mutex_wait_us":66,"thread_start_us":87,"threads_started":1}
I20260926 16:36:11.776013 29903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:11.769235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:42845 (local address 127.0.0.1:51218)
0926 16:36:11.769580 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:11.769632 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:11.769758 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:11.771281 (+ 1523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:11.771284 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:11.771302 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:36:11.771394 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:11.771399 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:11.772736 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:11.772740 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:11.773969 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:11.773977 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:11.775057 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:11.775061 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:11.775149 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:11.775168 (+ 19us) client_negotiation.cc:769] Sending connection context
0926 16:36:11.775234 (+ 66us) client_negotiation.cc:240] Negotiation successful
0926 16:36:11.775291 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":109,"thread_start_us":94,"threads_started":1}
I20260926 16:36:11.778673 29692 tablet_service.cc:1467] Processing CreateTablet for tablet c72f918147d04c968b2880b702726abc (DEFAULT_TABLE table=test-table [id=f609c585ddaf412b9f25cb62d881a157]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:36:11.778689 29826 tablet_service.cc:1467] Processing CreateTablet for tablet c72f918147d04c968b2880b702726abc (DEFAULT_TABLE table=test-table [id=f609c585ddaf412b9f25cb62d881a157]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:36:11.778923 29558 tablet_service.cc:1467] Processing CreateTablet for tablet c72f918147d04c968b2880b702726abc (DEFAULT_TABLE table=test-table [id=f609c585ddaf412b9f25cb62d881a157]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:36:11.779651 29692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c72f918147d04c968b2880b702726abc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:11.779695 29826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c72f918147d04c968b2880b702726abc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:11.779717 29558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c72f918147d04c968b2880b702726abc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:11.785684 29909 tablet_bootstrap.cc:492] T c72f918147d04c968b2880b702726abc P 3f339b8f27424a95b1f16a3fe71a25e0: Bootstrap starting.
I20260926 16:36:11.787084 29911 tablet_bootstrap.cc:492] T c72f918147d04c968b2880b702726abc P d3776302e9614e7c8a50bfb0ec5684af: Bootstrap starting.
I20260926 16:36:11.787523 29910 tablet_bootstrap.cc:492] T c72f918147d04c968b2880b702726abc P f85a53e3e81748aeb5af3bd28c01516d: Bootstrap starting.
I20260926 16:36:11.788200 29909 tablet_bootstrap.cc:654] T c72f918147d04c968b2880b702726abc P 3f339b8f27424a95b1f16a3fe71a25e0: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:11.788898 29909 log.cc:824] T c72f918147d04c968b2880b702726abc P 3f339b8f27424a95b1f16a3fe71a25e0: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:11.789119 29911 tablet_bootstrap.cc:654] T c72f918147d04c968b2880b702726abc P d3776302e9614e7c8a50bfb0ec5684af: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:11.789651 29910 tablet_bootstrap.cc:654] T c72f918147d04c968b2880b702726abc P f85a53e3e81748aeb5af3bd28c01516d: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:11.790611 29911 log.cc:824] T c72f918147d04c968b2880b702726abc P d3776302e9614e7c8a50bfb0ec5684af: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:11.790814 29910 log.cc:824] T c72f918147d04c968b2880b702726abc P f85a53e3e81748aeb5af3bd28c01516d: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:11.790934 29909 tablet_bootstrap.cc:492] T c72f918147d04c968b2880b702726abc P 3f339b8f27424a95b1f16a3fe71a25e0: No bootstrap required, opened a new log
I20260926 16:36:11.791193 29909 ts_tablet_manager.cc:1397] T c72f918147d04c968b2880b702726abc P 3f339b8f27424a95b1f16a3fe71a25e0: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260926 16:36:11.792577 29910 tablet_bootstrap.cc:492] T c72f918147d04c968b2880b702726abc P f85a53e3e81748aeb5af3bd28c01516d: No bootstrap required, opened a new log
I20260926 16:36:11.792577 29911 tablet_bootstrap.cc:492] T c72f918147d04c968b2880b702726abc P d3776302e9614e7c8a50bfb0ec5684af: No bootstrap required, opened a new log
I20260926 16:36:11.792753 29911 ts_tablet_manager.cc:1397] T c72f918147d04c968b2880b702726abc P d3776302e9614e7c8a50bfb0ec5684af: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 16:36:11.792753 29910 ts_tablet_manager.cc:1397] T c72f918147d04c968b2880b702726abc P f85a53e3e81748aeb5af3bd28c01516d: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260926 16:36:11.794157 29909 raft_consensus.cc:348] T c72f918147d04c968b2880b702726abc P 3f339b8f27424a95b1f16a3fe71a25e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3776302e9614e7c8a50bfb0ec5684af" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42845 } } peers { permanent_uuid: "3f339b8f27424a95b1f16a3fe71a25e0" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 41227 } } peers { permanent_uuid: "f85a53e3e81748aeb5af3bd28c01516d" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 40441 } }
I20260926 16:36:11.794494 29909 raft_consensus.cc:374] T c72f918147d04c968b2880b702726abc P 3f339b8f27424a95b1f16a3fe71a25e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:36:11.794570 29909 raft_consensus.cc:729] T c72f918147d04c968b2880b702726abc P 3f339b8f27424a95b1f16a3fe71a25e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f339b8f27424a95b1f16a3fe71a25e0, State: Initialized, Role: FOLLOWER
I20260926 16:36:11.794998 29909 consensus_queue.cc:260] T c72f918147d04c968b2880b702726abc P 3f339b8f27424a95b1f16a3fe71a25e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3776302e9614e7c8a50bfb0ec5684af" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42845 } } peers { permanent_uuid: "3f339b8f27424a95b1f16a3fe71a25e0" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 41227 } } peers { permanent_uuid: "f85a53e3e81748aeb5af3bd28c01516d" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 40441 } }
I20260926 16:36:11.795583 29758 heartbeater.cc:503] Master 127.11.8.254:43631 was elected leader, sending a full tablet report...
I20260926 16:36:11.796469 29910 raft_consensus.cc:348] T c72f918147d04c968b2880b702726abc P f85a53e3e81748aeb5af3bd28c01516d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3776302e9614e7c8a50bfb0ec5684af" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42845 } } peers { permanent_uuid: "3f339b8f27424a95b1f16a3fe71a25e0" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 41227 } } peers { permanent_uuid: "f85a53e3e81748aeb5af3bd28c01516d" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 40441 } }
I20260926 16:36:11.796450 29911 raft_consensus.cc:348] T c72f918147d04c968b2880b702726abc P d3776302e9614e7c8a50bfb0ec5684af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3776302e9614e7c8a50bfb0ec5684af" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42845 } } peers { permanent_uuid: "3f339b8f27424a95b1f16a3fe71a25e0" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 41227 } } peers { permanent_uuid: "f85a53e3e81748aeb5af3bd28c01516d" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 40441 } }
I20260926 16:36:11.796697 29911 raft_consensus.cc:374] T c72f918147d04c968b2880b702726abc P d3776302e9614e7c8a50bfb0ec5684af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:36:11.796766 29910 raft_consensus.cc:374] T c72f918147d04c968b2880b702726abc P f85a53e3e81748aeb5af3bd28c01516d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:36:11.796787 29911 raft_consensus.cc:729] T c72f918147d04c968b2880b702726abc P d3776302e9614e7c8a50bfb0ec5684af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3776302e9614e7c8a50bfb0ec5684af, State: Initialized, Role: FOLLOWER
I20260926 16:36:11.796808 29909 ts_tablet_manager.cc:1428] T c72f918147d04c968b2880b702726abc P 3f339b8f27424a95b1f16a3fe71a25e0: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260926 16:36:11.796857 29910 raft_consensus.cc:729] T c72f918147d04c968b2880b702726abc P f85a53e3e81748aeb5af3bd28c01516d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f85a53e3e81748aeb5af3bd28c01516d, State: Initialized, Role: FOLLOWER
I20260926 16:36:11.797284 29911 consensus_queue.cc:260] T c72f918147d04c968b2880b702726abc P d3776302e9614e7c8a50bfb0ec5684af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3776302e9614e7c8a50bfb0ec5684af" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42845 } } peers { permanent_uuid: "3f339b8f27424a95b1f16a3fe71a25e0" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 41227 } } peers { permanent_uuid: "f85a53e3e81748aeb5af3bd28c01516d" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 40441 } }
I20260926 16:36:11.797284 29910 consensus_queue.cc:260] T c72f918147d04c968b2880b702726abc P f85a53e3e81748aeb5af3bd28c01516d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3776302e9614e7c8a50bfb0ec5684af" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42845 } } peers { permanent_uuid: "3f339b8f27424a95b1f16a3fe71a25e0" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 41227 } } peers { permanent_uuid: "f85a53e3e81748aeb5af3bd28c01516d" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 40441 } }
I20260926 16:36:11.798153 29910 ts_tablet_manager.cc:1428] T c72f918147d04c968b2880b702726abc P f85a53e3e81748aeb5af3bd28c01516d: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260926 16:36:11.798197 29911 ts_tablet_manager.cc:1428] T c72f918147d04c968b2880b702726abc P d3776302e9614e7c8a50bfb0ec5684af: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260926 16:36:11.798544 29624 heartbeater.cc:503] Master 127.11.8.254:43631 was elected leader, sending a full tablet report...
I20260926 16:36:11.799355 29892 heartbeater.cc:503] Master 127.11.8.254:43631 was elected leader, sending a full tablet report...
W20260926 16:36:11.801914 29625 tablet.cc:2369] T c72f918147d04c968b2880b702726abc P d3776302e9614e7c8a50bfb0ec5684af: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 16:36:11.979663 29893 tablet.cc:2369] T c72f918147d04c968b2880b702726abc P f85a53e3e81748aeb5af3bd28c01516d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 16:36:12.021490 29759 tablet.cc:2369] T c72f918147d04c968b2880b702726abc P 3f339b8f27424a95b1f16a3fe71a25e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:36:12.085364 29917 raft_consensus.cc:482] T c72f918147d04c968b2880b702726abc P d3776302e9614e7c8a50bfb0ec5684af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:36:12.085553 29917 raft_consensus.cc:504] T c72f918147d04c968b2880b702726abc P d3776302e9614e7c8a50bfb0ec5684af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3776302e9614e7c8a50bfb0ec5684af" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42845 } } peers { permanent_uuid: "3f339b8f27424a95b1f16a3fe71a25e0" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 41227 } } peers { permanent_uuid: "f85a53e3e81748aeb5af3bd28c01516d" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 40441 } }
I20260926 16:36:12.086683 29917 leader_election.cc:288] T c72f918147d04c968b2880b702726abc P d3776302e9614e7c8a50bfb0ec5684af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f339b8f27424a95b1f16a3fe71a25e0 (127.11.8.194:41227), f85a53e3e81748aeb5af3bd28c01516d (127.11.8.195:40441)
I20260926 16:36:12.090456 29907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:12.086990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:42213 (local address 127.11.8.194:41227)
0926 16:36:12.087133 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:12.087139 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:12.087319 (+ 180us) server_negotiation.cc:407] Connection header received
0926 16:36:12.087502 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:12.087505 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:12.087586 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:12.087666 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:36:12.087955 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:12.088451 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:12.089504 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:12.089880 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:12.090010 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:12.090062 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:12.090243 (+ 181us) server_negotiation.cc:299] Negotiation successful
0926 16:36:12.090330 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 16:36:12.090488 29908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:12.087052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:38217 (local address 127.11.8.195:40441)
0926 16:36:12.087205 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:12.087209 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:12.087375 (+ 166us) server_negotiation.cc:407] Connection header received
0926 16:36:12.087535 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:12.087542 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:12.087621 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:12.087701 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:36:12.087976 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:12.088465 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:12.089519 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:12.089921 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:12.090046 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:12.090098 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:12.090298 (+ 200us) server_negotiation.cc:299] Negotiation successful
0926 16:36:12.090365 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 16:36:12.090510 29918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:12.086883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:41227 (local address 127.11.8.193:42213)
0926 16:36:12.087213 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:12.087229 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:12.087367 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:12.087685 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:12.087688 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:12.087703 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:36:12.087841 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:12.087846 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:12.088558 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:12.088563 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:12.089378 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:12.089386 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:12.090002 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:12.090005 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:12.090100 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:12.090126 (+ 26us) client_negotiation.cc:769] Sending connection context
0926 16:36:12.090218 (+ 92us) client_negotiation.cc:240] Negotiation successful
0926 16:36:12.090318 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":1,"spinlock_wait_cycles":11904,"thread_start_us":101,"threads_started":1}
I20260926 16:36:12.090530 29919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:12.086934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:40441 (local address 127.11.8.193:38217)
0926 16:36:12.087272 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:12.087283 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:12.087415 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:12.087710 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:12.087713 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:12.087748 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:36:12.087867 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:12.087872 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:12.088569 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:12.088576 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:12.089397 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:12.089404 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:12.090037 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:12.090041 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:12.090148 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:12.090176 (+ 28us) client_negotiation.cc:769] Sending connection context
0926 16:36:12.090261 (+ 85us) client_negotiation.cc:240] Negotiation successful
0926 16:36:12.090369 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"mutex_wait_us":27,"spinlock_wait_cycles":768,"thread_start_us":108,"threads_started":1}
I20260926 16:36:12.091212 29846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c72f918147d04c968b2880b702726abc" candidate_uuid: "d3776302e9614e7c8a50bfb0ec5684af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f85a53e3e81748aeb5af3bd28c01516d" is_pre_election: true
I20260926 16:36:12.091207 29712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c72f918147d04c968b2880b702726abc" candidate_uuid: "d3776302e9614e7c8a50bfb0ec5684af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f339b8f27424a95b1f16a3fe71a25e0" is_pre_election: true
I20260926 16:36:12.091502 29846 raft_consensus.cc:2454] T c72f918147d04c968b2880b702726abc P f85a53e3e81748aeb5af3bd28c01516d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3776302e9614e7c8a50bfb0ec5684af in term 0.
I20260926 16:36:12.091502 29712 raft_consensus.cc:2454] T c72f918147d04c968b2880b702726abc P 3f339b8f27424a95b1f16a3fe71a25e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3776302e9614e7c8a50bfb0ec5684af in term 0.
I20260926 16:36:12.092026 29513 leader_election.cc:302] T c72f918147d04c968b2880b702726abc P d3776302e9614e7c8a50bfb0ec5684af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3776302e9614e7c8a50bfb0ec5684af, f85a53e3e81748aeb5af3bd28c01516d; no voters:
I20260926 16:36:12.092314 29917 raft_consensus.cc:2781] T c72f918147d04c968b2880b702726abc P d3776302e9614e7c8a50bfb0ec5684af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:36:12.092414 29917 raft_consensus.cc:482] T c72f918147d04c968b2880b702726abc P d3776302e9614e7c8a50bfb0ec5684af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:36:12.092466 29917 raft_consensus.cc:3037] T c72f918147d04c968b2880b702726abc P d3776302e9614e7c8a50bfb0ec5684af [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:12.094065 29917 raft_consensus.cc:504] T c72f918147d04c968b2880b702726abc P d3776302e9614e7c8a50bfb0ec5684af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3776302e9614e7c8a50bfb0ec5684af" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42845 } } peers { permanent_uuid: "3f339b8f27424a95b1f16a3fe71a25e0" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 41227 } } peers { permanent_uuid: "f85a53e3e81748aeb5af3bd28c01516d" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 40441 } }
I20260926 16:36:12.094573 29917 leader_election.cc:288] T c72f918147d04c968b2880b702726abc P d3776302e9614e7c8a50bfb0ec5684af [CANDIDATE]: Term 1 election: Requested vote from peers 3f339b8f27424a95b1f16a3fe71a25e0 (127.11.8.194:41227), f85a53e3e81748aeb5af3bd28c01516d (127.11.8.195:40441)
I20260926 16:36:12.094929 29712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c72f918147d04c968b2880b702726abc" candidate_uuid: "d3776302e9614e7c8a50bfb0ec5684af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f339b8f27424a95b1f16a3fe71a25e0"
I20260926 16:36:12.095080 29712 raft_consensus.cc:3037] T c72f918147d04c968b2880b702726abc P 3f339b8f27424a95b1f16a3fe71a25e0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:12.096207 29846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c72f918147d04c968b2880b702726abc" candidate_uuid: "d3776302e9614e7c8a50bfb0ec5684af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f85a53e3e81748aeb5af3bd28c01516d"
I20260926 16:36:12.096306 29712 raft_consensus.cc:2454] T c72f918147d04c968b2880b702726abc P 3f339b8f27424a95b1f16a3fe71a25e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3776302e9614e7c8a50bfb0ec5684af in term 1.
I20260926 16:36:12.096342 29846 raft_consensus.cc:3037] T c72f918147d04c968b2880b702726abc P f85a53e3e81748aeb5af3bd28c01516d [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:12.096673 29514 leader_election.cc:302] T c72f918147d04c968b2880b702726abc P d3776302e9614e7c8a50bfb0ec5684af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f339b8f27424a95b1f16a3fe71a25e0, d3776302e9614e7c8a50bfb0ec5684af; no voters:
I20260926 16:36:12.096868 29917 raft_consensus.cc:2781] T c72f918147d04c968b2880b702726abc P d3776302e9614e7c8a50bfb0ec5684af [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:36:12.097071 29917 raft_consensus.cc:686] T c72f918147d04c968b2880b702726abc P d3776302e9614e7c8a50bfb0ec5684af [term 1 LEADER]: Becoming Leader. State: Replica: d3776302e9614e7c8a50bfb0ec5684af, State: Running, Role: LEADER
I20260926 16:36:12.097429 29917 consensus_queue.cc:237] T c72f918147d04c968b2880b702726abc P d3776302e9614e7c8a50bfb0ec5684af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3776302e9614e7c8a50bfb0ec5684af" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42845 } } peers { permanent_uuid: "3f339b8f27424a95b1f16a3fe71a25e0" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 41227 } } peers { permanent_uuid: "f85a53e3e81748aeb5af3bd28c01516d" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 40441 } }
I20260926 16:36:12.097690 29846 raft_consensus.cc:2454] T c72f918147d04c968b2880b702726abc P f85a53e3e81748aeb5af3bd28c01516d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3776302e9614e7c8a50bfb0ec5684af in term 1.
I20260926 16:36:12.100457 29433 catalog_manager.cc:5495] T c72f918147d04c968b2880b702726abc P d3776302e9614e7c8a50bfb0ec5684af reported cstate change: term changed from 0 to 1, leader changed from <none> to d3776302e9614e7c8a50bfb0ec5684af (127.11.8.193). New cstate: current_term: 1 leader_uuid: "d3776302e9614e7c8a50bfb0ec5684af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3776302e9614e7c8a50bfb0ec5684af" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 42845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f339b8f27424a95b1f16a3fe71a25e0" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 41227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f85a53e3e81748aeb5af3bd28c01516d" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 40441 } health_report { overall_health: UNKNOWN } } }
I20260926 16:36:12.136183 29904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:12.132572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51234 (local address 127.11.8.193:42845)
0926 16:36:12.132712 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:12.132716 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:12.132732 (+ 16us) server_negotiation.cc:407] Connection header received
0926 16:36:12.132827 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:12.132829 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:12.132883 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:12.133009 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:36:12.133263 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:12.133767 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:12.134672 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:12.134898 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:12.135008 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:12.135314 (+ 306us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:36:12.135738 (+ 424us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:36:12.135903 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:12.136007 (+ 104us) server_negotiation.cc:299] Negotiation successful
0926 16:36:12.136047 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 16:36:12.141897 29846 raft_consensus.cc:1261] T c72f918147d04c968b2880b702726abc P f85a53e3e81748aeb5af3bd28c01516d [term 1 FOLLOWER]: Refusing update from remote peer d3776302e9614e7c8a50bfb0ec5684af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:36:12.141897 29712 raft_consensus.cc:1261] T c72f918147d04c968b2880b702726abc P 3f339b8f27424a95b1f16a3fe71a25e0 [term 1 FOLLOWER]: Refusing update from remote peer d3776302e9614e7c8a50bfb0ec5684af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:36:12.142654 29920 consensus_queue.cc:1035] T c72f918147d04c968b2880b702726abc P d3776302e9614e7c8a50bfb0ec5684af [LEADER]: Connected to new peer: Peer: permanent_uuid: "f85a53e3e81748aeb5af3bd28c01516d" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 40441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:36:12.142827 29917 consensus_queue.cc:1035] T c72f918147d04c968b2880b702726abc P d3776302e9614e7c8a50bfb0ec5684af [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f339b8f27424a95b1f16a3fe71a25e0" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 41227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:36:12.149878 29923 mvcc.cc:205] Tried to move back new op lower bound from 7333644583486320640 to 7333644583314075648. Current Snapshot: MvccSnapshot[applied={T|T < 7333644583486320640}]
I20260926 16:36:12.151893 29925 mvcc.cc:205] Tried to move back new op lower bound from 7333644583486320640 to 7333644583314075648. Current Snapshot: MvccSnapshot[applied={T|T < 7333644583486320640 or (T in {7333644583486320640})}]
I20260926 16:36:12.152336 29924 mvcc.cc:205] Tried to move back new op lower bound from 7333644583486320640 to 7333644583314075648. Current Snapshot: MvccSnapshot[applied={T|T < 7333644583486320640}]
I20260926 16:36:12.158757 29433 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36550:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:36:12.158931 29433 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36550:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:36:12.161060 29433 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b007db5545174d659f7fef0fb24486df: Sending DeleteTablet for 3 replicas of tablet c72f918147d04c968b2880b702726abc
I20260926 16:36:12.161753 29558 tablet_service.cc:1514] Processing DeleteTablet for tablet c72f918147d04c968b2880b702726abc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:36:12 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:51218
I20260926 16:36:12.162010 29826 tablet_service.cc:1514] Processing DeleteTablet for tablet c72f918147d04c968b2880b702726abc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:36:12 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:51204
I20260926 16:36:12.162202 29692 tablet_service.cc:1514] Processing DeleteTablet for tablet c72f918147d04c968b2880b702726abc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:36:12 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:57330
I20260926 16:36:12.162465 29934 tablet_replica.cc:330] T c72f918147d04c968b2880b702726abc P f85a53e3e81748aeb5af3bd28c01516d: stopping tablet replica
I20260926 16:36:12.162595 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 29495
I20260926 16:36:12.163376 29935 tablet_replica.cc:330] T c72f918147d04c968b2880b702726abc P 3f339b8f27424a95b1f16a3fe71a25e0: stopping tablet replica
I20260926 16:36:12.163679 29935 raft_consensus.cc:2229] T c72f918147d04c968b2880b702726abc P 3f339b8f27424a95b1f16a3fe71a25e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:36:12.163968 29935 raft_consensus.cc:2258] T c72f918147d04c968b2880b702726abc P 3f339b8f27424a95b1f16a3fe71a25e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:36:12.164812 29934 raft_consensus.cc:2229] T c72f918147d04c968b2880b702726abc P f85a53e3e81748aeb5af3bd28c01516d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:36:12.165112 29934 raft_consensus.cc:2258] T c72f918147d04c968b2880b702726abc P f85a53e3e81748aeb5af3bd28c01516d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:36:12.166050 29935 ts_tablet_manager.cc:1905] T c72f918147d04c968b2880b702726abc P 3f339b8f27424a95b1f16a3fe71a25e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:36:12.166551 29934 ts_tablet_manager.cc:1905] T c72f918147d04c968b2880b702726abc P f85a53e3e81748aeb5af3bd28c01516d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:36:12.168696 29935 ts_tablet_manager.cc:1918] T c72f918147d04c968b2880b702726abc P 3f339b8f27424a95b1f16a3fe71a25e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:36:12.168805 29935 log.cc:1196] T c72f918147d04c968b2880b702726abc P 3f339b8f27424a95b1f16a3fe71a25e0: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-1/wal/wals/c72f918147d04c968b2880b702726abc
I20260926 16:36:12.169098 29935 ts_tablet_manager.cc:1939] T c72f918147d04c968b2880b702726abc P 3f339b8f27424a95b1f16a3fe71a25e0: Deleting consensus metadata
W20260926 16:36:12.169931 29420 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.8.193:42845 (error 108)
I20260926 16:36:12.170094 29934 ts_tablet_manager.cc:1918] T c72f918147d04c968b2880b702726abc P f85a53e3e81748aeb5af3bd28c01516d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:36:12.170186 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 29629
I20260926 16:36:12.170209 29934 log.cc:1196] T c72f918147d04c968b2880b702726abc P f85a53e3e81748aeb5af3bd28c01516d: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790440324823152-11299-0/minicluster-data/ts-2/wal/wals/c72f918147d04c968b2880b702726abc
I20260926 16:36:12.170516 29934 ts_tablet_manager.cc:1939] T c72f918147d04c968b2880b702726abc P f85a53e3e81748aeb5af3bd28c01516d: Deleting consensus metadata
I20260926 16:36:12.171502 29906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:12.171139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:42845 (local address 127.0.0.1:51240)
0926 16:36:12.171325 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:12.171395 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.193:42845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260926 16:36:12.171607 29419 catalog_manager.cc:4841] TS f85a53e3e81748aeb5af3bd28c01516d (127.11.8.195:40441): tablet c72f918147d04c968b2880b702726abc (table test-table [id=f609c585ddaf412b9f25cb62d881a157]) successfully deleted
W20260926 16:36:12.171706 29420 catalog_manager.cc:4584] TS d3776302e9614e7c8a50bfb0ec5684af (127.11.8.193:42845): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c72f918147d04c968b2880b702726abc: Network error: Client connection negotiation failed: client connection to 127.11.8.193:42845: connect: Connection refused (error 111)
I20260926 16:36:12.176795 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 29763
I20260926 16:36:12.178133 29906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:12.177867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:41227 (local address 127.0.0.1:57334)
0926 16:36:12.178028 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:12.178070 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.194:41227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260926 16:36:12.182564 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 29401
2026-09-26T16:36:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2495 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-26T16:36:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:36:12Z Disabled control of system clock
I20260926 16:36:12.212229 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:45973
--webserver_interface=127.11.8.254
--webserver_port=0
--builtin_ntp_servers=127.11.8.212:45285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:45973
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 16:36:12.319859 29940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:12.320091 29940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:12.320125 29940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:12.324354 29940 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:36:12.324416 29940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:12.324432 29940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:12.324450 29940 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:36:12.324465 29940 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:36:12.329478 29940 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:45285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:45973
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:45973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:12.330470 29940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:12.331763 29940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 16:36:12.339974 29940 server_base.cc:1034] running on GCE node
W20260926 16:36:12.339936 29945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:12.339903 29946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:12.339932 29948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:12.340703 29940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:12.341718 29940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:12.342882 29940 hybrid_clock.cc:648] HybridClock initialized: now 1790440572342863 us; error 32 us; skew 500 ppm
I20260926 16:36:12.344604 29940 webserver.cc:460] Webserver started at http://127.11.8.254:32795/ using document root <none> and password file <none>
I20260926 16:36:12.345103 29940 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:12.345158 29940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:12.345310 29940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:12.347268 29940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "f6b56fccdc3145b48e9db389d83753e5"
format_stamp: "Formatted at 2026-09-26 16:36:12 on dist-test-slave-2zvq"
server_key: "955717d7b68957626fb968ca7a2e01eb"
server_key_iv: "7e9deaaf2dbee2f52a935b3132c47aa9"
server_key_version: "encryptionkey@0"
I20260926 16:36:12.347749 29940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "f6b56fccdc3145b48e9db389d83753e5"
format_stamp: "Formatted at 2026-09-26 16:36:12 on dist-test-slave-2zvq"
server_key: "955717d7b68957626fb968ca7a2e01eb"
server_key_iv: "7e9deaaf2dbee2f52a935b3132c47aa9"
server_key_version: "encryptionkey@0"
I20260926 16:36:12.351084 29940 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:36:12.353207 29954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:12.354439 29940 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260926 16:36:12.354528 29940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "f6b56fccdc3145b48e9db389d83753e5"
format_stamp: "Formatted at 2026-09-26 16:36:12 on dist-test-slave-2zvq"
server_key: "955717d7b68957626fb968ca7a2e01eb"
server_key_iv: "7e9deaaf2dbee2f52a935b3132c47aa9"
server_key_version: "encryptionkey@0"
I20260926 16:36:12.354605 29940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:12.369385 29940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:12.369947 29940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:12.370081 29940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:12.376631 29940 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:45973
I20260926 16:36:12.376641 30006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:45973 every 8 connection(s)
I20260926 16:36:12.377496 29940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:36:12.380215 30007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:12.385499 30007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6b56fccdc3145b48e9db389d83753e5: Bootstrap starting.
I20260926 16:36:12.387637 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 29940
I20260926 16:36:12.387734 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:36:12.388017 11299 external_mini_cluster.cc:1428] Setting key bf7d3dfd9ca37d48459342e050042bc1
I20260926 16:36:12.388036 30007 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6b56fccdc3145b48e9db389d83753e5: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:12.388762 30007 log.cc:824] T 00000000000000000000000000000000 P f6b56fccdc3145b48e9db389d83753e5: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:12.390826 30007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6b56fccdc3145b48e9db389d83753e5: No bootstrap required, opened a new log
I20260926 16:36:12.395066 30007 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6b56fccdc3145b48e9db389d83753e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6b56fccdc3145b48e9db389d83753e5" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 45973 } }
I20260926 16:36:12.395299 30007 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6b56fccdc3145b48e9db389d83753e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:36:12.395370 30007 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6b56fccdc3145b48e9db389d83753e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6b56fccdc3145b48e9db389d83753e5, State: Initialized, Role: FOLLOWER
I20260926 16:36:12.395785 30007 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6b56fccdc3145b48e9db389d83753e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6b56fccdc3145b48e9db389d83753e5" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 45973 } }
I20260926 16:36:12.395956 30007 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6b56fccdc3145b48e9db389d83753e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:36:12.396070 30007 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6b56fccdc3145b48e9db389d83753e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:36:12.396206 30007 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6b56fccdc3145b48e9db389d83753e5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:12.397084 30011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:12.389118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49208 (local address 127.11.8.254:45973)
0926 16:36:12.389744 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:12.389760 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:12.389795 (+ 35us) server_negotiation.cc:407] Connection header received
0926 16:36:12.390511 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:12.390530 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:12.390849 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:12.391187 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:12.391516 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:12.392453 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:12.393653 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:12.394117 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:12.394238 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:12.395120 (+ 882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:12.395125 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:12.395138 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:36:12.395687 (+ 549us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:12.395747 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:36:12.395769 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:12.395845 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:12.396034 (+ 189us) server_negotiation.cc:299] Negotiation successful
0926 16:36:12.396340 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":123,"threads_started":1}
I20260926 16:36:12.397104 30007 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6b56fccdc3145b48e9db389d83753e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6b56fccdc3145b48e9db389d83753e5" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 45973 } }
I20260926 16:36:12.397452 30007 leader_election.cc:302] T 00000000000000000000000000000000 P f6b56fccdc3145b48e9db389d83753e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6b56fccdc3145b48e9db389d83753e5; no voters:
I20260926 16:36:12.397742 30007 leader_election.cc:288] T 00000000000000000000000000000000 P f6b56fccdc3145b48e9db389d83753e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:36:12.397833 30012 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6b56fccdc3145b48e9db389d83753e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:36:12.398030 30012 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6b56fccdc3145b48e9db389d83753e5 [term 1 LEADER]: Becoming Leader. State: Replica: f6b56fccdc3145b48e9db389d83753e5, State: Running, Role: LEADER
I20260926 16:36:12.398423 30012 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6b56fccdc3145b48e9db389d83753e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6b56fccdc3145b48e9db389d83753e5" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 45973 } }
I20260926 16:36:12.399049 30007 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6b56fccdc3145b48e9db389d83753e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:36:12.400465 30014 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6b56fccdc3145b48e9db389d83753e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6b56fccdc3145b48e9db389d83753e5. Latest consensus state: current_term: 1 leader_uuid: "f6b56fccdc3145b48e9db389d83753e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6b56fccdc3145b48e9db389d83753e5" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 45973 } } }
I20260926 16:36:12.400503 30013 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6b56fccdc3145b48e9db389d83753e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6b56fccdc3145b48e9db389d83753e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6b56fccdc3145b48e9db389d83753e5" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 45973 } } }
I20260926 16:36:12.400610 30014 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6b56fccdc3145b48e9db389d83753e5 [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:12.400614 30013 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6b56fccdc3145b48e9db389d83753e5 [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:12.401448 30023 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:36:12.404134 30023 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:36:12.409322 30023 catalog_manager.cc:1347] Generated new cluster ID: 9da46c095de642158bfb577f156be5d9
I20260926 16:36:12.409467 30023 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:36:12.449709 30023 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:36:12.450994 30023 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:36:12.457499 30023 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6b56fccdc3145b48e9db389d83753e5: Generated new TSK 0
I20260926 16:36:12.458310 30023 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:36:12.471131 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.193:0
--local_ip_for_outbound_sockets=127.11.8.193
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:45973
--builtin_ntp_servers=127.11.8.212:45285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 16:36:12.579770 30031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:12.580022 30031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:12.580060 30031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:12.584085 30031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:12.584141 30031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:12.584193 30031 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.193
I20260926 16:36:12.588989 30031 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:45285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:45973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.193
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:12.590080 30031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:12.591456 30031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:12.601265 30036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:12.601236 30039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:12.601248 30037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:12.601907 30031 server_base.cc:1034] running on GCE node
I20260926 16:36:12.602384 30031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:12.603055 30031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:12.604240 30031 hybrid_clock.cc:648] HybridClock initialized: now 1790440572604233 us; error 30 us; skew 500 ppm
I20260926 16:36:12.606086 30031 webserver.cc:460] Webserver started at http://127.11.8.193:45325/ using document root <none> and password file <none>
I20260926 16:36:12.606736 30031 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:12.606830 30031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:12.607040 30031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:12.609066 30031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-0/data/instance:
uuid: "a5cd6d744fbc4e4aaf86e6d44e9ec921"
format_stamp: "Formatted at 2026-09-26 16:36:12 on dist-test-slave-2zvq"
server_key: "9c49683c1de4832ba97d298309174f15"
server_key_iv: "17d42b128412c3e6ff53944b17552369"
server_key_version: "encryptionkey@0"
I20260926 16:36:12.609647 30031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance:
uuid: "a5cd6d744fbc4e4aaf86e6d44e9ec921"
format_stamp: "Formatted at 2026-09-26 16:36:12 on dist-test-slave-2zvq"
server_key: "9c49683c1de4832ba97d298309174f15"
server_key_iv: "17d42b128412c3e6ff53944b17552369"
server_key_version: "encryptionkey@0"
I20260926 16:36:12.613267 30031 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260926 16:36:12.615726 30045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:12.616942 30031 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 16:36:12.617071 30031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-0/wal
uuid: "a5cd6d744fbc4e4aaf86e6d44e9ec921"
format_stamp: "Formatted at 2026-09-26 16:36:12 on dist-test-slave-2zvq"
server_key: "9c49683c1de4832ba97d298309174f15"
server_key_iv: "17d42b128412c3e6ff53944b17552369"
server_key_version: "encryptionkey@0"
I20260926 16:36:12.617180 30031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:12.627547 30031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:12.628198 30031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:12.628394 30031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:12.628937 30031 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:36:12.629897 30031 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:36:12.629978 30031 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:12.630070 30031 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:36:12.630123 30031 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:12.638803 30031 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.193:46457
I20260926 16:36:12.638833 30158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.193:46457 every 8 connection(s)
I20260926 16:36:12.639696 30031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
I20260926 16:36:12.647178 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 30031
I20260926 16:36:12.647327 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance
I20260926 16:36:12.647663 11299 external_mini_cluster.cc:1428] Setting key b663421637cea901835703a9233d653f
I20260926 16:36:12.648872 30011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:12.641479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:35369 (local address 127.11.8.254:45973)
0926 16:36:12.641654 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:12.641659 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:12.642745 (+ 1086us) server_negotiation.cc:407] Connection header received
0926 16:36:12.644063 (+ 1318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:12.644065 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:12.644110 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:12.644193 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:12.645064 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:12.645594 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:12.646780 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:12.647008 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:12.647095 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:12.648378 (+ 1283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:12.648381 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:12.648383 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:36:12.648394 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:12.648414 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:36:12.648421 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:12.648475 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:12.648656 (+ 181us) server_negotiation.cc:299] Negotiation successful
0926 16:36:12.648715 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 16:36:12.649796 30161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:12.641874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:45973 (local address 127.11.8.193:35369)
0926 16:36:12.642583 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:12.642633 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:12.643881 (+ 1248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:12.644364 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:12.644370 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:12.644450 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:36:12.644905 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:12.644918 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:12.645719 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:12.645721 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:12.646649 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:12.646658 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:12.647185 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:12.647191 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:12.647412 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:12.648082 (+ 670us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:36:12.648086 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:36:12.648138 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:36:12.648200 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:36:12.648503 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:36:12.648514 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:36:12.648520 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 16:36:12.648628 (+ 108us) client_negotiation.cc:240] Negotiation successful
0926 16:36:12.648842 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":125,"threads_started":1}
I20260926 16:36:12.649897 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.194:0
--local_ip_for_outbound_sockets=127.11.8.194
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:45973
--builtin_ntp_servers=127.11.8.212:45285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 16:36:12.651154 30159 heartbeater.cc:347] Connected to a master server at 127.11.8.254:45973
I20260926 16:36:12.651391 30159 heartbeater.cc:464] Registering TS with master...
I20260926 16:36:12.651870 30159 heartbeater.cc:511] Master 127.11.8.254:45973 requested a full tablet report, sending...
I20260926 16:36:12.653415 29969 ts_manager.cc:194] Registered new tserver with Master: a5cd6d744fbc4e4aaf86e6d44e9ec921 (127.11.8.193:46457)
I20260926 16:36:12.655131 29969 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.8.193:35369
W20260926 16:36:12.758847 30162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:12.759130 30162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:12.759169 30162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:12.763072 30162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:12.763120 30162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:12.763172 30162 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.194
I20260926 16:36:12.767792 30162 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:45285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:45973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30162
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.194
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:12.768725 30162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:12.769989 30162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:12.779311 30170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:12.779347 30168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:12.779336 30167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:12.779560 30162 server_base.cc:1034] running on GCE node
I20260926 16:36:12.780074 30162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:12.780696 30162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:12.781888 30162 hybrid_clock.cc:648] HybridClock initialized: now 1790440572781852 us; error 45 us; skew 500 ppm
I20260926 16:36:12.783738 30162 webserver.cc:460] Webserver started at http://127.11.8.194:42689/ using document root <none> and password file <none>
I20260926 16:36:12.784310 30162 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:12.784372 30162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:12.784617 30162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:12.786659 30162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-1/data/instance:
uuid: "c59e67bd13fc4767bbdb2fa2dbc34872"
format_stamp: "Formatted at 2026-09-26 16:36:12 on dist-test-slave-2zvq"
server_key: "19e0d85d8aab76161cc6d475e7e7eef4"
server_key_iv: "d6e1a6ccc8bf67c175d60dc9df1697ea"
server_key_version: "encryptionkey@0"
I20260926 16:36:12.787204 30162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance:
uuid: "c59e67bd13fc4767bbdb2fa2dbc34872"
format_stamp: "Formatted at 2026-09-26 16:36:12 on dist-test-slave-2zvq"
server_key: "19e0d85d8aab76161cc6d475e7e7eef4"
server_key_iv: "d6e1a6ccc8bf67c175d60dc9df1697ea"
server_key_version: "encryptionkey@0"
I20260926 16:36:12.790747 30162 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 16:36:12.792972 30176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:12.794070 30162 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:36:12.794195 30162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-1/wal
uuid: "c59e67bd13fc4767bbdb2fa2dbc34872"
format_stamp: "Formatted at 2026-09-26 16:36:12 on dist-test-slave-2zvq"
server_key: "19e0d85d8aab76161cc6d475e7e7eef4"
server_key_iv: "d6e1a6ccc8bf67c175d60dc9df1697ea"
server_key_version: "encryptionkey@0"
I20260926 16:36:12.794323 30162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:12.827315 30162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:12.828016 30162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:12.828213 30162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:12.828804 30162 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:36:12.829814 30162 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:36:12.829882 30162 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:12.829974 30162 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:36:12.830026 30162 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:12.838660 30162 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.194:39793
I20260926 16:36:12.838737 30289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.194:39793 every 8 connection(s)
I20260926 16:36:12.839548 30162 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
I20260926 16:36:12.846726 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 30162
I20260926 16:36:12.846858 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance
I20260926 16:36:12.847178 11299 external_mini_cluster.cc:1428] Setting key 33caf277a0815c3c36ecfe5fcdcdc4de
I20260926 16:36:12.848479 30011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:12.841298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:55469 (local address 127.11.8.254:45973)
0926 16:36:12.841435 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:12.841439 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:12.842479 (+ 1040us) server_negotiation.cc:407] Connection header received
0926 16:36:12.843801 (+ 1322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:12.843804 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:12.843856 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:12.843929 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:12.844721 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:12.845239 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:12.846343 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:12.846676 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:12.846793 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:12.848013 (+ 1220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:12.848016 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:12.848019 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:36:12.848031 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:12.848052 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:36:12.848062 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:12.848143 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:12.848259 (+ 116us) server_negotiation.cc:299] Negotiation successful
0926 16:36:12.848310 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 16:36:12.849177 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.195:0
--local_ip_for_outbound_sockets=127.11.8.195
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:45973
--builtin_ntp_servers=127.11.8.212:45285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 16:36:12.849272 30292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:12.841685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:45973 (local address 127.11.8.194:55469)
0926 16:36:12.842343 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:12.842383 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:12.843628 (+ 1245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:12.844085 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:12.844091 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:12.844170 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:36:12.844606 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:12.844617 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:12.845336 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:12.845339 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:12.846172 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:12.846181 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:12.846840 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:12.846845 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:12.847046 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:12.847717 (+ 671us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:36:12.847721 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:36:12.847772 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:36:12.847847 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:36:12.848139 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:36:12.848143 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:36:12.848147 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 16:36:12.848228 (+ 81us) client_negotiation.cc:240] Negotiation successful
0926 16:36:12.848472 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":108,"threads_started":1}
I20260926 16:36:12.850225 30290 heartbeater.cc:347] Connected to a master server at 127.11.8.254:45973
I20260926 16:36:12.850493 30290 heartbeater.cc:464] Registering TS with master...
I20260926 16:36:12.850984 30290 heartbeater.cc:511] Master 127.11.8.254:45973 requested a full tablet report, sending...
I20260926 16:36:12.851953 29969 ts_manager.cc:194] Registered new tserver with Master: c59e67bd13fc4767bbdb2fa2dbc34872 (127.11.8.194:39793)
I20260926 16:36:12.852490 29969 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.8.194:55469
W20260926 16:36:12.958768 30293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:12.959035 30293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:12.959097 30293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:12.963027 30293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:12.963105 30293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:12.963196 30293 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.195
I20260926 16:36:12.967897 30293 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:45285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:45973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30293
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.195
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:12.968890 30293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:12.970232 30293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:12.979460 30299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:12.979455 30298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:12.979511 30293 server_base.cc:1034] running on GCE node
W20260926 16:36:12.979432 30301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:12.979995 30293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:12.980597 30293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:12.981756 30293 hybrid_clock.cc:648] HybridClock initialized: now 1790440572981739 us; error 27 us; skew 500 ppm
I20260926 16:36:12.983608 30293 webserver.cc:460] Webserver started at http://127.11.8.195:44971/ using document root <none> and password file <none>
I20260926 16:36:12.984169 30293 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:12.984230 30293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:12.984457 30293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:12.986491 30293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-2/data/instance:
uuid: "55527fe476ec47c3aac002e65f011e1c"
format_stamp: "Formatted at 2026-09-26 16:36:12 on dist-test-slave-2zvq"
server_key: "b1b52da5a25207a3fb35193219f7ecdf"
server_key_iv: "090155a08effe6096034ec428af18c28"
server_key_version: "encryptionkey@0"
I20260926 16:36:12.986995 30293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance:
uuid: "55527fe476ec47c3aac002e65f011e1c"
format_stamp: "Formatted at 2026-09-26 16:36:12 on dist-test-slave-2zvq"
server_key: "b1b52da5a25207a3fb35193219f7ecdf"
server_key_iv: "090155a08effe6096034ec428af18c28"
server_key_version: "encryptionkey@0"
I20260926 16:36:12.990501 30293 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260926 16:36:12.992781 30307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:12.994016 30293 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:36:12.994155 30293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-2/wal
uuid: "55527fe476ec47c3aac002e65f011e1c"
format_stamp: "Formatted at 2026-09-26 16:36:12 on dist-test-slave-2zvq"
server_key: "b1b52da5a25207a3fb35193219f7ecdf"
server_key_iv: "090155a08effe6096034ec428af18c28"
server_key_version: "encryptionkey@0"
I20260926 16:36:12.994259 30293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:13.024174 30293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:13.024880 30293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:13.025091 30293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:13.025702 30293 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:36:13.026651 30293 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:36:13.026697 30293 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:13.026762 30293 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:36:13.026794 30293 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:13.035705 30293 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.195:33593
I20260926 16:36:13.035733 30420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.195:33593 every 8 connection(s)
I20260926 16:36:13.036615 30293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
I20260926 16:36:13.045532 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 30293
I20260926 16:36:13.045663 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance
I20260926 16:36:13.045655 30011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:13.038477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:49831 (local address 127.11.8.254:45973)
0926 16:36:13.038614 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:13.038618 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:13.039687 (+ 1069us) server_negotiation.cc:407] Connection header received
0926 16:36:13.041080 (+ 1393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:13.041084 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:13.041141 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:13.041250 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:13.042132 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:13.042691 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:13.043814 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:13.044043 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:13.044159 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:13.045166 (+ 1007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:13.045170 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:13.045173 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:36:13.045191 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:13.045209 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:36:13.045216 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:13.045291 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:13.045448 (+ 157us) server_negotiation.cc:299] Negotiation successful
0926 16:36:13.045514 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 16:36:13.045998 11299 external_mini_cluster.cc:1428] Setting key 9b9f078f88782d89d11f331833ddc6f5
I20260926 16:36:13.046626 30423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:13.038910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:45973 (local address 127.11.8.195:49831)
0926 16:36:13.039542 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:13.039581 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:13.040873 (+ 1292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:13.041421 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:13.041428 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:13.041510 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:36:13.041989 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:13.042002 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:13.042832 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:13.042835 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:13.043685 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:13.043694 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:13.044192 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:13.044196 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:13.044358 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:13.044920 (+ 562us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:36:13.044921 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:36:13.044959 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:36:13.045017 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:36:13.045304 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:36:13.045316 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:36:13.045327 (+ 11us) client_negotiation.cc:769] Sending connection context
0926 16:36:13.045448 (+ 121us) client_negotiation.cc:240] Negotiation successful
0926 16:36:13.045702 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":101,"threads_started":1}
I20260926 16:36:13.047793 30421 heartbeater.cc:347] Connected to a master server at 127.11.8.254:45973
I20260926 16:36:13.048023 30421 heartbeater.cc:464] Registering TS with master...
I20260926 16:36:13.048588 30421 heartbeater.cc:511] Master 127.11.8.254:45973 requested a full tablet report, sending...
I20260926 16:36:13.049667 29969 ts_manager.cc:194] Registered new tserver with Master: 55527fe476ec47c3aac002e65f011e1c (127.11.8.195:33593)
I20260926 16:36:13.050230 29969 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.8.195:49831
I20260926 16:36:13.059784 11299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 16:36:13.065454 30011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:13.061838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49222 (local address 127.11.8.254:45973)
0926 16:36:13.061977 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:13.061981 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:13.062111 (+ 130us) server_negotiation.cc:407] Connection header received
0926 16:36:13.062199 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:13.062201 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:13.062237 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:13.062370 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:13.062593 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:13.063077 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:13.063866 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:13.064065 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:13.064138 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:13.064961 (+ 823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:13.064964 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:13.064966 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:36:13.064976 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:13.064991 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:36:13.065005 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:13.065102 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:13.065223 (+ 121us) server_negotiation.cc:299] Negotiation successful
0926 16:36:13.065280 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 16:36:13.068223 29969 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49222:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:36:13.071148 29969 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:36:13.085232 30437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:13.080023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52578 (local address 127.11.8.194:39793)
0926 16:36:13.080389 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:13.080402 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:13.080428 (+ 26us) server_negotiation.cc:407] Connection header received
0926 16:36:13.080505 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:13.080510 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:13.080709 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:13.080875 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:36:13.081285 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:13.082025 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:13.083765 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:13.084422 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:13.084524 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:13.084712 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:13.084965 (+ 253us) server_negotiation.cc:299] Negotiation successful
0926 16:36:13.085058 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":98,"threads_started":1}
I20260926 16:36:13.085230 30432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:13.079187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:46457 (local address 127.0.0.1:41532)
0926 16:36:13.079482 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:13.079524 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:13.079637 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:13.080904 (+ 1267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:13.080912 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:13.080938 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:36:13.081165 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:13.081172 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:13.082604 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:13.082607 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:13.083811 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:13.083818 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:13.084594 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:13.084598 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:13.084726 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:13.084807 (+ 81us) client_negotiation.cc:769] Sending connection context
0926 16:36:13.084925 (+ 118us) client_negotiation.cc:240] Negotiation successful
0926 16:36:13.085052 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":172,"thread_start_us":58,"threads_started":1}
I20260926 16:36:13.085237 30435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:13.079876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:39793 (local address 127.0.0.1:52578)
0926 16:36:13.080094 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:13.080107 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:13.080192 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:13.081017 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:13.081020 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:13.081036 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:36:13.081168 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:13.081173 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:13.082137 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:13.082143 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:13.083630 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:13.083640 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:13.084558 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:13.084561 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:13.084721 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:13.084808 (+ 87us) client_negotiation.cc:769] Sending connection context
0926 16:36:13.084921 (+ 113us) client_negotiation.cc:240] Negotiation successful
0926 16:36:13.085056 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":180,"thread_start_us":96,"threads_started":1}
I20260926 16:36:13.086046 30433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:13.079335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41532 (local address 127.11.8.193:46457)
0926 16:36:13.080400 (+ 1065us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:13.080414 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:13.080433 (+ 19us) server_negotiation.cc:407] Connection header received
0926 16:36:13.080505 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:13.080510 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:13.080709 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:13.080888 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:36:13.081293 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:13.082480 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:13.083926 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:13.084448 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:13.084573 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:13.084764 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:13.085840 (+ 1076us) server_negotiation.cc:299] Negotiation successful
0926 16:36:13.085930 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":961,"thread_start_us":97,"threads_started":1}
I20260926 16:36:13.086367 30434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:13.079462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:33593 (local address 127.0.0.1:50740)
0926 16:36:13.079701 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:13.079714 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:13.079801 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:13.080967 (+ 1166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:13.080970 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:13.080982 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:36:13.081166 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:13.081172 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:13.082658 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:13.082662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:13.083805 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:13.083814 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:13.085942 (+ 2128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:13.085946 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:13.086053 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:13.086076 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 16:36:13.086123 (+ 47us) client_negotiation.cc:240] Negotiation successful
0926 16:36:13.086186 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":148,"thread_start_us":85,"threads_started":1}
I20260926 16:36:13.087122 30436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:13.079942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50740 (local address 127.11.8.195:33593)
0926 16:36:13.080476 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:13.080488 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:13.080511 (+ 23us) server_negotiation.cc:407] Connection header received
0926 16:36:13.080588 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:13.080593 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:13.080794 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:13.081304 (+ 510us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:36:13.081438 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:13.082525 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:13.084835 (+ 2310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:13.085799 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:13.086603 (+ 804us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:13.086812 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:13.086875 (+ 63us) server_negotiation.cc:299] Negotiation successful
0926 16:36:13.086962 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":156,"threads_started":1}
I20260926 16:36:13.088441 30093 tablet_service.cc:1467] Processing CreateTablet for tablet ffc414b8cf9846539288e9b0542a8a9f (DEFAULT_TABLE table=test-table [id=1f71b990a5164d72a09b5f30a2ba0e02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:36:13.088441 30224 tablet_service.cc:1467] Processing CreateTablet for tablet ffc414b8cf9846539288e9b0542a8a9f (DEFAULT_TABLE table=test-table [id=1f71b990a5164d72a09b5f30a2ba0e02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:36:13.089551 30224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffc414b8cf9846539288e9b0542a8a9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:13.089551 30093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffc414b8cf9846539288e9b0542a8a9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:13.096508 30355 tablet_service.cc:1467] Processing CreateTablet for tablet ffc414b8cf9846539288e9b0542a8a9f (DEFAULT_TABLE table=test-table [id=1f71b990a5164d72a09b5f30a2ba0e02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:36:13.096956 30438 tablet_bootstrap.cc:492] T ffc414b8cf9846539288e9b0542a8a9f P a5cd6d744fbc4e4aaf86e6d44e9ec921: Bootstrap starting.
I20260926 16:36:13.097586 30355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffc414b8cf9846539288e9b0542a8a9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:13.098299 30439 tablet_bootstrap.cc:492] T ffc414b8cf9846539288e9b0542a8a9f P c59e67bd13fc4767bbdb2fa2dbc34872: Bootstrap starting.
I20260926 16:36:13.100898 30438 tablet_bootstrap.cc:654] T ffc414b8cf9846539288e9b0542a8a9f P a5cd6d744fbc4e4aaf86e6d44e9ec921: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:13.101745 30438 log.cc:824] T ffc414b8cf9846539288e9b0542a8a9f P a5cd6d744fbc4e4aaf86e6d44e9ec921: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:13.103091 30439 tablet_bootstrap.cc:654] T ffc414b8cf9846539288e9b0542a8a9f P c59e67bd13fc4767bbdb2fa2dbc34872: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:13.103730 30438 tablet_bootstrap.cc:492] T ffc414b8cf9846539288e9b0542a8a9f P a5cd6d744fbc4e4aaf86e6d44e9ec921: No bootstrap required, opened a new log
I20260926 16:36:13.103895 30439 log.cc:824] T ffc414b8cf9846539288e9b0542a8a9f P c59e67bd13fc4767bbdb2fa2dbc34872: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:13.103941 30438 ts_tablet_manager.cc:1397] T ffc414b8cf9846539288e9b0542a8a9f P a5cd6d744fbc4e4aaf86e6d44e9ec921: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260926 16:36:13.104984 30442 tablet_bootstrap.cc:492] T ffc414b8cf9846539288e9b0542a8a9f P 55527fe476ec47c3aac002e65f011e1c: Bootstrap starting.
I20260926 16:36:13.105865 30439 tablet_bootstrap.cc:492] T ffc414b8cf9846539288e9b0542a8a9f P c59e67bd13fc4767bbdb2fa2dbc34872: No bootstrap required, opened a new log
I20260926 16:36:13.106058 30439 ts_tablet_manager.cc:1397] T ffc414b8cf9846539288e9b0542a8a9f P c59e67bd13fc4767bbdb2fa2dbc34872: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260926 16:36:13.107059 30442 tablet_bootstrap.cc:654] T ffc414b8cf9846539288e9b0542a8a9f P 55527fe476ec47c3aac002e65f011e1c: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:13.107800 30442 log.cc:824] T ffc414b8cf9846539288e9b0542a8a9f P 55527fe476ec47c3aac002e65f011e1c: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:13.108296 30438 raft_consensus.cc:348] T ffc414b8cf9846539288e9b0542a8a9f P a5cd6d744fbc4e4aaf86e6d44e9ec921 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5cd6d744fbc4e4aaf86e6d44e9ec921" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46457 } } peers { permanent_uuid: "c59e67bd13fc4767bbdb2fa2dbc34872" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 39793 } } peers { permanent_uuid: "55527fe476ec47c3aac002e65f011e1c" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33593 } }
I20260926 16:36:13.108597 30438 raft_consensus.cc:374] T ffc414b8cf9846539288e9b0542a8a9f P a5cd6d744fbc4e4aaf86e6d44e9ec921 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:36:13.108659 30438 raft_consensus.cc:729] T ffc414b8cf9846539288e9b0542a8a9f P a5cd6d744fbc4e4aaf86e6d44e9ec921 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5cd6d744fbc4e4aaf86e6d44e9ec921, State: Initialized, Role: FOLLOWER
I20260926 16:36:13.109124 30438 consensus_queue.cc:260] T ffc414b8cf9846539288e9b0542a8a9f P a5cd6d744fbc4e4aaf86e6d44e9ec921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5cd6d744fbc4e4aaf86e6d44e9ec921" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46457 } } peers { permanent_uuid: "c59e67bd13fc4767bbdb2fa2dbc34872" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 39793 } } peers { permanent_uuid: "55527fe476ec47c3aac002e65f011e1c" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33593 } }
I20260926 16:36:13.109764 30159 heartbeater.cc:503] Master 127.11.8.254:45973 was elected leader, sending a full tablet report...
I20260926 16:36:13.109745 30439 raft_consensus.cc:348] T ffc414b8cf9846539288e9b0542a8a9f P c59e67bd13fc4767bbdb2fa2dbc34872 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5cd6d744fbc4e4aaf86e6d44e9ec921" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46457 } } peers { permanent_uuid: "c59e67bd13fc4767bbdb2fa2dbc34872" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 39793 } } peers { permanent_uuid: "55527fe476ec47c3aac002e65f011e1c" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33593 } }
I20260926 16:36:13.110010 30439 raft_consensus.cc:374] T ffc414b8cf9846539288e9b0542a8a9f P c59e67bd13fc4767bbdb2fa2dbc34872 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:36:13.110034 30438 ts_tablet_manager.cc:1428] T ffc414b8cf9846539288e9b0542a8a9f P a5cd6d744fbc4e4aaf86e6d44e9ec921: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260926 16:36:13.110069 30439 raft_consensus.cc:729] T ffc414b8cf9846539288e9b0542a8a9f P c59e67bd13fc4767bbdb2fa2dbc34872 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c59e67bd13fc4767bbdb2fa2dbc34872, State: Initialized, Role: FOLLOWER
I20260926 16:36:13.110500 30439 consensus_queue.cc:260] T ffc414b8cf9846539288e9b0542a8a9f P c59e67bd13fc4767bbdb2fa2dbc34872 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5cd6d744fbc4e4aaf86e6d44e9ec921" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46457 } } peers { permanent_uuid: "c59e67bd13fc4767bbdb2fa2dbc34872" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 39793 } } peers { permanent_uuid: "55527fe476ec47c3aac002e65f011e1c" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33593 } }
I20260926 16:36:13.111089 30290 heartbeater.cc:503] Master 127.11.8.254:45973 was elected leader, sending a full tablet report...
I20260926 16:36:13.112118 30442 tablet_bootstrap.cc:492] T ffc414b8cf9846539288e9b0542a8a9f P 55527fe476ec47c3aac002e65f011e1c: No bootstrap required, opened a new log
I20260926 16:36:13.112321 30442 ts_tablet_manager.cc:1397] T ffc414b8cf9846539288e9b0542a8a9f P 55527fe476ec47c3aac002e65f011e1c: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260926 16:36:13.112591 30439 ts_tablet_manager.cc:1428] T ffc414b8cf9846539288e9b0542a8a9f P c59e67bd13fc4767bbdb2fa2dbc34872: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260926 16:36:13.116007 30442 raft_consensus.cc:348] T ffc414b8cf9846539288e9b0542a8a9f P 55527fe476ec47c3aac002e65f011e1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5cd6d744fbc4e4aaf86e6d44e9ec921" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46457 } } peers { permanent_uuid: "c59e67bd13fc4767bbdb2fa2dbc34872" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 39793 } } peers { permanent_uuid: "55527fe476ec47c3aac002e65f011e1c" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33593 } }
I20260926 16:36:13.116183 30442 raft_consensus.cc:374] T ffc414b8cf9846539288e9b0542a8a9f P 55527fe476ec47c3aac002e65f011e1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:36:13.116251 30442 raft_consensus.cc:729] T ffc414b8cf9846539288e9b0542a8a9f P 55527fe476ec47c3aac002e65f011e1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55527fe476ec47c3aac002e65f011e1c, State: Initialized, Role: FOLLOWER
I20260926 16:36:13.116606 30442 consensus_queue.cc:260] T ffc414b8cf9846539288e9b0542a8a9f P 55527fe476ec47c3aac002e65f011e1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5cd6d744fbc4e4aaf86e6d44e9ec921" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46457 } } peers { permanent_uuid: "c59e67bd13fc4767bbdb2fa2dbc34872" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 39793 } } peers { permanent_uuid: "55527fe476ec47c3aac002e65f011e1c" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33593 } }
I20260926 16:36:13.117158 30421 heartbeater.cc:503] Master 127.11.8.254:45973 was elected leader, sending a full tablet report...
I20260926 16:36:13.117412 30442 ts_tablet_manager.cc:1428] T ffc414b8cf9846539288e9b0542a8a9f P 55527fe476ec47c3aac002e65f011e1c: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260926 16:36:13.140817 30160 tablet.cc:2369] T ffc414b8cf9846539288e9b0542a8a9f P a5cd6d744fbc4e4aaf86e6d44e9ec921: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 16:36:13.287585 30422 tablet.cc:2369] T ffc414b8cf9846539288e9b0542a8a9f P 55527fe476ec47c3aac002e65f011e1c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 16:36:13.340619 30291 tablet.cc:2369] T ffc414b8cf9846539288e9b0542a8a9f P c59e67bd13fc4767bbdb2fa2dbc34872: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:36:13.367681 30444 raft_consensus.cc:482] T ffc414b8cf9846539288e9b0542a8a9f P a5cd6d744fbc4e4aaf86e6d44e9ec921 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:36:13.367936 30444 raft_consensus.cc:504] T ffc414b8cf9846539288e9b0542a8a9f P a5cd6d744fbc4e4aaf86e6d44e9ec921 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5cd6d744fbc4e4aaf86e6d44e9ec921" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46457 } } peers { permanent_uuid: "c59e67bd13fc4767bbdb2fa2dbc34872" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 39793 } } peers { permanent_uuid: "55527fe476ec47c3aac002e65f011e1c" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33593 } }
I20260926 16:36:13.369097 30444 leader_election.cc:288] T ffc414b8cf9846539288e9b0542a8a9f P a5cd6d744fbc4e4aaf86e6d44e9ec921 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c59e67bd13fc4767bbdb2fa2dbc34872 (127.11.8.194:39793), 55527fe476ec47c3aac002e65f011e1c (127.11.8.195:33593)
I20260926 16:36:13.372872 30437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:13.369441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:41291 (local address 127.11.8.194:39793)
0926 16:36:13.369575 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:13.369580 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:13.369744 (+ 164us) server_negotiation.cc:407] Connection header received
0926 16:36:13.369912 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:13.369915 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:13.369965 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:13.370051 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:36:13.370351 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:13.370850 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:13.371948 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:13.372320 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:13.372454 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:13.372500 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:13.372688 (+ 188us) server_negotiation.cc:299] Negotiation successful
0926 16:36:13.372754 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 16:36:13.372891 30447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:13.369322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:39793 (local address 127.11.8.193:41291)
0926 16:36:13.369646 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:13.369661 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:13.369793 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:13.370064 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:13.370066 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:13.370081 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:36:13.370229 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:13.370235 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:13.370967 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:13.370972 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:13.371816 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:13.371824 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:13.372454 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:13.372457 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:13.372553 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:13.372581 (+ 28us) client_negotiation.cc:769] Sending connection context
0926 16:36:13.372673 (+ 92us) client_negotiation.cc:240] Negotiation successful
0926 16:36:13.372739 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":86,"thread_start_us":73,"threads_started":1}
I20260926 16:36:13.373081 30436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:13.369489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:47521 (local address 127.11.8.195:33593)
0926 16:36:13.369616 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:13.369621 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:13.369797 (+ 176us) server_negotiation.cc:407] Connection header received
0926 16:36:13.369965 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:13.369967 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:13.370006 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:13.370089 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:36:13.370388 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:13.370861 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:13.371934 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:13.372316 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:13.372440 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:13.372489 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:13.372909 (+ 420us) server_negotiation.cc:299] Negotiation successful
0926 16:36:13.372978 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 16:36:13.373087 30448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:13.369347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:33593 (local address 127.11.8.193:47521)
0926 16:36:13.369710 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:13.369721 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:13.369838 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:13.370101 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:13.370104 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:13.370119 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:36:13.370253 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:13.370259 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:13.370970 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:13.370974 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:13.371808 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:13.371817 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:13.372440 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:13.372444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:13.372774 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:13.372792 (+ 18us) client_negotiation.cc:769] Sending connection context
0926 16:36:13.372884 (+ 92us) client_negotiation.cc:240] Negotiation successful
0926 16:36:13.372949 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":4,"thread_start_us":106,"threads_started":1}
I20260926 16:36:13.373481 30244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffc414b8cf9846539288e9b0542a8a9f" candidate_uuid: "a5cd6d744fbc4e4aaf86e6d44e9ec921" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c59e67bd13fc4767bbdb2fa2dbc34872" is_pre_election: true
I20260926 16:36:13.373628 30375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffc414b8cf9846539288e9b0542a8a9f" candidate_uuid: "a5cd6d744fbc4e4aaf86e6d44e9ec921" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55527fe476ec47c3aac002e65f011e1c" is_pre_election: true
I20260926 16:36:13.373809 30244 raft_consensus.cc:2454] T ffc414b8cf9846539288e9b0542a8a9f P c59e67bd13fc4767bbdb2fa2dbc34872 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5cd6d744fbc4e4aaf86e6d44e9ec921 in term 0.
I20260926 16:36:13.373853 30375 raft_consensus.cc:2454] T ffc414b8cf9846539288e9b0542a8a9f P 55527fe476ec47c3aac002e65f011e1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5cd6d744fbc4e4aaf86e6d44e9ec921 in term 0.
I20260926 16:36:13.374379 30048 leader_election.cc:302] T ffc414b8cf9846539288e9b0542a8a9f P a5cd6d744fbc4e4aaf86e6d44e9ec921 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 55527fe476ec47c3aac002e65f011e1c, a5cd6d744fbc4e4aaf86e6d44e9ec921, c59e67bd13fc4767bbdb2fa2dbc34872; no voters:
I20260926 16:36:13.374651 30444 raft_consensus.cc:2781] T ffc414b8cf9846539288e9b0542a8a9f P a5cd6d744fbc4e4aaf86e6d44e9ec921 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:36:13.374744 30444 raft_consensus.cc:482] T ffc414b8cf9846539288e9b0542a8a9f P a5cd6d744fbc4e4aaf86e6d44e9ec921 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:36:13.374804 30444 raft_consensus.cc:3037] T ffc414b8cf9846539288e9b0542a8a9f P a5cd6d744fbc4e4aaf86e6d44e9ec921 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:13.376020 30444 raft_consensus.cc:504] T ffc414b8cf9846539288e9b0542a8a9f P a5cd6d744fbc4e4aaf86e6d44e9ec921 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5cd6d744fbc4e4aaf86e6d44e9ec921" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46457 } } peers { permanent_uuid: "c59e67bd13fc4767bbdb2fa2dbc34872" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 39793 } } peers { permanent_uuid: "55527fe476ec47c3aac002e65f011e1c" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33593 } }
I20260926 16:36:13.376430 30444 leader_election.cc:288] T ffc414b8cf9846539288e9b0542a8a9f P a5cd6d744fbc4e4aaf86e6d44e9ec921 [CANDIDATE]: Term 1 election: Requested vote from peers c59e67bd13fc4767bbdb2fa2dbc34872 (127.11.8.194:39793), 55527fe476ec47c3aac002e65f011e1c (127.11.8.195:33593)
I20260926 16:36:13.376726 30244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffc414b8cf9846539288e9b0542a8a9f" candidate_uuid: "a5cd6d744fbc4e4aaf86e6d44e9ec921" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c59e67bd13fc4767bbdb2fa2dbc34872"
I20260926 16:36:13.376849 30375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffc414b8cf9846539288e9b0542a8a9f" candidate_uuid: "a5cd6d744fbc4e4aaf86e6d44e9ec921" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55527fe476ec47c3aac002e65f011e1c"
I20260926 16:36:13.376871 30244 raft_consensus.cc:3037] T ffc414b8cf9846539288e9b0542a8a9f P c59e67bd13fc4767bbdb2fa2dbc34872 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:13.376959 30375 raft_consensus.cc:3037] T ffc414b8cf9846539288e9b0542a8a9f P 55527fe476ec47c3aac002e65f011e1c [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:13.378114 30375 raft_consensus.cc:2454] T ffc414b8cf9846539288e9b0542a8a9f P 55527fe476ec47c3aac002e65f011e1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5cd6d744fbc4e4aaf86e6d44e9ec921 in term 1.
I20260926 16:36:13.378114 30244 raft_consensus.cc:2454] T ffc414b8cf9846539288e9b0542a8a9f P c59e67bd13fc4767bbdb2fa2dbc34872 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5cd6d744fbc4e4aaf86e6d44e9ec921 in term 1.
I20260926 16:36:13.378448 30049 leader_election.cc:302] T ffc414b8cf9846539288e9b0542a8a9f P a5cd6d744fbc4e4aaf86e6d44e9ec921 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a5cd6d744fbc4e4aaf86e6d44e9ec921, c59e67bd13fc4767bbdb2fa2dbc34872; no voters:
I20260926 16:36:13.378633 30444 raft_consensus.cc:2781] T ffc414b8cf9846539288e9b0542a8a9f P a5cd6d744fbc4e4aaf86e6d44e9ec921 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:36:13.378840 30444 raft_consensus.cc:686] T ffc414b8cf9846539288e9b0542a8a9f P a5cd6d744fbc4e4aaf86e6d44e9ec921 [term 1 LEADER]: Becoming Leader. State: Replica: a5cd6d744fbc4e4aaf86e6d44e9ec921, State: Running, Role: LEADER
I20260926 16:36:13.379161 30444 consensus_queue.cc:237] T ffc414b8cf9846539288e9b0542a8a9f P a5cd6d744fbc4e4aaf86e6d44e9ec921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5cd6d744fbc4e4aaf86e6d44e9ec921" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46457 } } peers { permanent_uuid: "c59e67bd13fc4767bbdb2fa2dbc34872" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 39793 } } peers { permanent_uuid: "55527fe476ec47c3aac002e65f011e1c" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33593 } }
I20260926 16:36:13.382534 29969 catalog_manager.cc:5495] T ffc414b8cf9846539288e9b0542a8a9f P a5cd6d744fbc4e4aaf86e6d44e9ec921 reported cstate change: term changed from 0 to 1, leader changed from <none> to a5cd6d744fbc4e4aaf86e6d44e9ec921 (127.11.8.193). New cstate: current_term: 1 leader_uuid: "a5cd6d744fbc4e4aaf86e6d44e9ec921" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5cd6d744fbc4e4aaf86e6d44e9ec921" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c59e67bd13fc4767bbdb2fa2dbc34872" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 39793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55527fe476ec47c3aac002e65f011e1c" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33593 } health_report { overall_health: UNKNOWN } } }
I20260926 16:36:13.455358 30433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:13.451935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41548 (local address 127.11.8.193:46457)
0926 16:36:13.452067 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:13.452071 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:13.452086 (+ 15us) server_negotiation.cc:407] Connection header received
0926 16:36:13.452182 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:13.452184 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:13.452237 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:13.452328 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:36:13.452567 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:13.453057 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:13.453892 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:13.454108 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:13.454189 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:13.454516 (+ 327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:36:13.454903 (+ 387us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:36:13.455018 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:13.455138 (+ 120us) server_negotiation.cc:299] Negotiation successful
0926 16:36:13.455196 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 16:36:13.461223 30375 raft_consensus.cc:1261] T ffc414b8cf9846539288e9b0542a8a9f P 55527fe476ec47c3aac002e65f011e1c [term 1 FOLLOWER]: Refusing update from remote peer a5cd6d744fbc4e4aaf86e6d44e9ec921: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:36:13.461223 30244 raft_consensus.cc:1261] T ffc414b8cf9846539288e9b0542a8a9f P c59e67bd13fc4767bbdb2fa2dbc34872 [term 1 FOLLOWER]: Refusing update from remote peer a5cd6d744fbc4e4aaf86e6d44e9ec921: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:36:13.462015 30449 consensus_queue.cc:1035] T ffc414b8cf9846539288e9b0542a8a9f P a5cd6d744fbc4e4aaf86e6d44e9ec921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55527fe476ec47c3aac002e65f011e1c" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:36:13.462225 30444 consensus_queue.cc:1035] T ffc414b8cf9846539288e9b0542a8a9f P a5cd6d744fbc4e4aaf86e6d44e9ec921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c59e67bd13fc4767bbdb2fa2dbc34872" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 39793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:36:13.468917 30452 mvcc.cc:205] Tried to move back new op lower bound from 7333644588889583616 to 7333644588563689472. Current Snapshot: MvccSnapshot[applied={T|T < 7333644588889583616}]
I20260926 16:36:13.469703 30454 mvcc.cc:205] Tried to move back new op lower bound from 7333644588889583616 to 7333644588563689472. Current Snapshot: MvccSnapshot[applied={T|T < 7333644588889583616}]
I20260926 16:36:13.477337 29969 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49222:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:36:13.477481 29969 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49222:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:36:13.480038 29969 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f6b56fccdc3145b48e9db389d83753e5: Sending DeleteTablet for 3 replicas of tablet ffc414b8cf9846539288e9b0542a8a9f
I20260926 16:36:13.480784 30093 tablet_service.cc:1514] Processing DeleteTablet for tablet ffc414b8cf9846539288e9b0542a8a9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:36:13 UTC) from {username='slave'} at 127.0.0.1:41532
I20260926 16:36:13.480798 30224 tablet_service.cc:1514] Processing DeleteTablet for tablet ffc414b8cf9846539288e9b0542a8a9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:36:13 UTC) from {username='slave'} at 127.0.0.1:52578
I20260926 16:36:13.480923 30355 tablet_service.cc:1514] Processing DeleteTablet for tablet ffc414b8cf9846539288e9b0542a8a9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:36:13 UTC) from {username='slave'} at 127.0.0.1:50740
I20260926 16:36:13.481431 30464 tablet_replica.cc:330] T ffc414b8cf9846539288e9b0542a8a9f P 55527fe476ec47c3aac002e65f011e1c: stopping tablet replica
I20260926 16:36:13.481657 30462 tablet_replica.cc:330] T ffc414b8cf9846539288e9b0542a8a9f P a5cd6d744fbc4e4aaf86e6d44e9ec921: stopping tablet replica
I20260926 16:36:13.481765 30464 raft_consensus.cc:2229] T ffc414b8cf9846539288e9b0542a8a9f P 55527fe476ec47c3aac002e65f011e1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:36:13.481956 30462 raft_consensus.cc:2229] T ffc414b8cf9846539288e9b0542a8a9f P a5cd6d744fbc4e4aaf86e6d44e9ec921 [term 1 LEADER]: Raft consensus shutting down.
I20260926 16:36:13.482074 30464 raft_consensus.cc:2258] T ffc414b8cf9846539288e9b0542a8a9f P 55527fe476ec47c3aac002e65f011e1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:36:13.482300 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 30031
I20260926 16:36:13.482338 30462 raft_consensus.cc:2258] T ffc414b8cf9846539288e9b0542a8a9f P a5cd6d744fbc4e4aaf86e6d44e9ec921 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:36:13.482659 30463 tablet_replica.cc:330] T ffc414b8cf9846539288e9b0542a8a9f P c59e67bd13fc4767bbdb2fa2dbc34872: stopping tablet replica
I20260926 16:36:13.482931 30463 raft_consensus.cc:2229] T ffc414b8cf9846539288e9b0542a8a9f P c59e67bd13fc4767bbdb2fa2dbc34872 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:36:13.483246 30463 raft_consensus.cc:2258] T ffc414b8cf9846539288e9b0542a8a9f P c59e67bd13fc4767bbdb2fa2dbc34872 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:36:13.485530 30464 ts_tablet_manager.cc:1905] T ffc414b8cf9846539288e9b0542a8a9f P 55527fe476ec47c3aac002e65f011e1c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:36:13.485687 30463 ts_tablet_manager.cc:1905] T ffc414b8cf9846539288e9b0542a8a9f P c59e67bd13fc4767bbdb2fa2dbc34872: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:36:13.488027 30463 ts_tablet_manager.cc:1918] T ffc414b8cf9846539288e9b0542a8a9f P c59e67bd13fc4767bbdb2fa2dbc34872: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:36:13.488131 30463 log.cc:1196] T ffc414b8cf9846539288e9b0542a8a9f P c59e67bd13fc4767bbdb2fa2dbc34872: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-1/wal/wals/ffc414b8cf9846539288e9b0542a8a9f
I20260926 16:36:13.488417 30463 ts_tablet_manager.cc:1939] T ffc414b8cf9846539288e9b0542a8a9f P c59e67bd13fc4767bbdb2fa2dbc34872: Deleting consensus metadata
I20260926 16:36:13.488983 30464 ts_tablet_manager.cc:1918] T ffc414b8cf9846539288e9b0542a8a9f P 55527fe476ec47c3aac002e65f011e1c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:36:13.489084 30464 log.cc:1196] T ffc414b8cf9846539288e9b0542a8a9f P 55527fe476ec47c3aac002e65f011e1c: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790440324823152-11299-0/minicluster-data/ts-2/wal/wals/ffc414b8cf9846539288e9b0542a8a9f
I20260926 16:36:13.489620 29958 catalog_manager.cc:4841] TS c59e67bd13fc4767bbdb2fa2dbc34872 (127.11.8.194:39793): tablet ffc414b8cf9846539288e9b0542a8a9f (table test-table [id=1f71b990a5164d72a09b5f30a2ba0e02]) successfully deleted
I20260926 16:36:13.489379 30464 ts_tablet_manager.cc:1939] T ffc414b8cf9846539288e9b0542a8a9f P 55527fe476ec47c3aac002e65f011e1c: Deleting consensus metadata
W20260926 16:36:13.490818 29958 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.8.193:46457 (error 108)
I20260926 16:36:13.490988 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 30162
I20260926 16:36:13.492623 29957 catalog_manager.cc:4841] TS 55527fe476ec47c3aac002e65f011e1c (127.11.8.195:33593): tablet ffc414b8cf9846539288e9b0542a8a9f (table test-table [id=1f71b990a5164d72a09b5f30a2ba0e02]) successfully deleted
I20260926 16:36:13.495040 30434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:13.494770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:46457 (local address 127.0.0.1:41558)
0926 16:36:13.494898 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:13.494970 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.193:46457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260926 16:36:13.495189 29958 catalog_manager.cc:4584] TS a5cd6d744fbc4e4aaf86e6d44e9ec921 (127.11.8.193:46457): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ffc414b8cf9846539288e9b0542a8a9f: Network error: Client connection negotiation failed: client connection to 127.11.8.193:46457: connect: Connection refused (error 111)
I20260926 16:36:13.496991 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 30293
I20260926 16:36:13.502614 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 29940
2026-09-26T16:36:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1318 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:36:13 dist-test-slave-2zvq krb5kdc[30468](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:36:13 dist-test-slave-2zvq krb5kdc[30468](info): set up 2 sockets
Sep 26 16:36:13 dist-test-slave-2zvq krb5kdc[30468](info): commencing operation
krb5kdc: starting...
W20260926 16:36:14.535925 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:36:14 dist-test-slave-2zvq krb5kdc[30468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:36:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:36:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
I20260926 16:36:14.653704 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:42265
--webserver_interface=127.11.8.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:44499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:42265
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:36:14.761209 30481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:14.761464 30481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:14.761502 30481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:14.765391 30481 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:36:14.765445 30481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:14.765465 30481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:14.765480 30481 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:36:14.765496 30481 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:36:14.770731 30481 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:44499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:42265
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:42265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:14.772112 30481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:14.773118 30481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:14.782320 30486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:14.782404 30489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:14.782349 30481 server_base.cc:1034] running on GCE node
W20260926 16:36:14.782361 30487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:14.783037 30481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:14.784049 30481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:14.785211 30481 hybrid_clock.cc:648] HybridClock initialized: now 1790440574785198 us; error 24 us; skew 500 ppm
Sep 26 16:36:14 dist-test-slave-2zvq krb5kdc[30468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440574, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:36:14.787972 30481 init.cc:377] Logged in from keytab as kudu/127.11.8.254@KRBTEST.COM (short username kudu)
I20260926 16:36:14.788794 30481 webserver.cc:460] Webserver started at http://127.11.8.254:36119/ using document root <none> and password file <none>
I20260926 16:36:14.789335 30481 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:14.789392 30481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:14.789631 30481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:14.791893 30481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "d602f22fa25f43e48a324b47992b9204"
format_stamp: "Formatted at 2026-09-26 16:36:14 on dist-test-slave-2zvq"
server_key: "d648fa3d055882a8ba2702c622d616be"
server_key_iv: "905c352c2adbbc98885612a137db84c5"
server_key_version: "encryptionkey@0"
I20260926 16:36:14.792384 30481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "d602f22fa25f43e48a324b47992b9204"
format_stamp: "Formatted at 2026-09-26 16:36:14 on dist-test-slave-2zvq"
server_key: "d648fa3d055882a8ba2702c622d616be"
server_key_iv: "905c352c2adbbc98885612a137db84c5"
server_key_version: "encryptionkey@0"
I20260926 16:36:14.795868 30481 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260926 16:36:14.798045 30496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:14.799355 30481 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260926 16:36:14.799450 30481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "d602f22fa25f43e48a324b47992b9204"
format_stamp: "Formatted at 2026-09-26 16:36:14 on dist-test-slave-2zvq"
server_key: "d648fa3d055882a8ba2702c622d616be"
server_key_iv: "905c352c2adbbc98885612a137db84c5"
server_key_version: "encryptionkey@0"
I20260926 16:36:14.799530 30481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:14.811712 30481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:14.814841 30481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:14.814985 30481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:14.822301 30481 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:42265
I20260926 16:36:14.822371 30548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:42265 every 8 connection(s)
I20260926 16:36:14.823342 30481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:36:14.826020 30549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:14.829523 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 30481
I20260926 16:36:14.829633 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:36:14.829919 11299 external_mini_cluster.cc:1428] Setting key fc62d0172f72a882900d28ec08fc3c94
I20260926 16:36:14.831606 30549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d602f22fa25f43e48a324b47992b9204: Bootstrap starting.
I20260926 16:36:14.833901 30549 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d602f22fa25f43e48a324b47992b9204: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:14.834671 30549 log.cc:824] T 00000000000000000000000000000000 P d602f22fa25f43e48a324b47992b9204: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:14.836797 30549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d602f22fa25f43e48a324b47992b9204: No bootstrap required, opened a new log
Sep 26 16:36:14 dist-test-slave-2zvq krb5kdc[30468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:36:14.841157 30549 raft_consensus.cc:348] T 00000000000000000000000000000000 P d602f22fa25f43e48a324b47992b9204 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d602f22fa25f43e48a324b47992b9204" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 42265 } }
I20260926 16:36:14.841389 30549 raft_consensus.cc:374] T 00000000000000000000000000000000 P d602f22fa25f43e48a324b47992b9204 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:36:14.841478 30549 raft_consensus.cc:729] T 00000000000000000000000000000000 P d602f22fa25f43e48a324b47992b9204 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d602f22fa25f43e48a324b47992b9204, State: Initialized, Role: FOLLOWER
I20260926 16:36:14.841935 30549 consensus_queue.cc:260] T 00000000000000000000000000000000 P d602f22fa25f43e48a324b47992b9204 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d602f22fa25f43e48a324b47992b9204" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 42265 } }
I20260926 16:36:14.842077 30549 raft_consensus.cc:388] T 00000000000000000000000000000000 P d602f22fa25f43e48a324b47992b9204 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:36:14.842161 30549 raft_consensus.cc:482] T 00000000000000000000000000000000 P d602f22fa25f43e48a324b47992b9204 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:36:14.842257 30549 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d602f22fa25f43e48a324b47992b9204 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:14.843267 30549 raft_consensus.cc:504] T 00000000000000000000000000000000 P d602f22fa25f43e48a324b47992b9204 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d602f22fa25f43e48a324b47992b9204" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 42265 } }
I20260926 16:36:14.843650 30549 leader_election.cc:302] T 00000000000000000000000000000000 P d602f22fa25f43e48a324b47992b9204 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d602f22fa25f43e48a324b47992b9204; no voters:
I20260926 16:36:14.843952 30549 leader_election.cc:288] T 00000000000000000000000000000000 P d602f22fa25f43e48a324b47992b9204 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:36:14.844051 30554 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d602f22fa25f43e48a324b47992b9204 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:36:14.844249 30554 raft_consensus.cc:686] T 00000000000000000000000000000000 P d602f22fa25f43e48a324b47992b9204 [term 1 LEADER]: Becoming Leader. State: Replica: d602f22fa25f43e48a324b47992b9204, State: Running, Role: LEADER
I20260926 16:36:14.844602 30554 consensus_queue.cc:237] T 00000000000000000000000000000000 P d602f22fa25f43e48a324b47992b9204 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d602f22fa25f43e48a324b47992b9204" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 42265 } }
I20260926 16:36:14.844764 30552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:14.831444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37130 (local address 127.11.8.254:42265)
0926 16:36:14.831831 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:14.831846 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:14.831879 (+ 33us) server_negotiation.cc:407] Connection header received
0926 16:36:14.832474 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:14.832489 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:14.832819 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:14.833121 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:14.834049 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:14.835342 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:14.836508 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:14.837035 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:14.837154 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:14.839799 (+ 2645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:14.839804 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:14.839816 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:14.839842 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:14.841610 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:14.842033 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:14.842039 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:14.842044 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:14.842189 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:14.842552 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:14.842554 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:14.842555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:14.842907 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:14.843052 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:14.843470 (+ 418us) server_negotiation.cc:299] Negotiation successful
0926 16:36:14.843794 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":125,"threads_started":1}
I20260926 16:36:14.845114 30549 sys_catalog.cc:564] T 00000000000000000000000000000000 P d602f22fa25f43e48a324b47992b9204 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:36:14.845739 30554 sys_catalog.cc:455] T 00000000000000000000000000000000 P d602f22fa25f43e48a324b47992b9204 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d602f22fa25f43e48a324b47992b9204. Latest consensus state: current_term: 1 leader_uuid: "d602f22fa25f43e48a324b47992b9204" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d602f22fa25f43e48a324b47992b9204" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 42265 } } }
I20260926 16:36:14.845855 30554 sys_catalog.cc:458] T 00000000000000000000000000000000 P d602f22fa25f43e48a324b47992b9204 [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:14.846484 30555 sys_catalog.cc:455] T 00000000000000000000000000000000 P d602f22fa25f43e48a324b47992b9204 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d602f22fa25f43e48a324b47992b9204" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d602f22fa25f43e48a324b47992b9204" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 42265 } } }
I20260926 16:36:14.846635 30555 sys_catalog.cc:458] T 00000000000000000000000000000000 P d602f22fa25f43e48a324b47992b9204 [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:14.847517 30559 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:36:14.849924 30559 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:36:14.855401 30559 catalog_manager.cc:1347] Generated new cluster ID: 81737b1178c64389bba5dbf3c2e71740
I20260926 16:36:14.855470 30559 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:36:14.884039 30559 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:36:14.885332 30559 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:36:14.890988 30559 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d602f22fa25f43e48a324b47992b9204: Generated new TSK 0
I20260926 16:36:14.891733 30559 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.8.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab.
Entry for principal kudu/127.11.8.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab.
I20260926 16:36:14.921680 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.193:0
--local_ip_for_outbound_sockets=127.11.8.193
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:42265
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:44499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:36:15.028616 30575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:15.028884 30575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:15.028957 30575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:15.032994 30575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:15.033077 30575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:15.033169 30575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.193
I20260926 16:36:15.038439 30575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:44499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.193.keytab
--principal=kudu/127.11.8.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:42265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.193
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:15.039490 30575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:15.040510 30575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:15.050313 30580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:15.050400 30583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:15.050400 30581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:15.050333 30575 server_base.cc:1034] running on GCE node
I20260926 16:36:15.051007 30575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:15.051630 30575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:15.052819 30575 hybrid_clock.cc:648] HybridClock initialized: now 1790440575052804 us; error 30 us; skew 500 ppm
Sep 26 16:36:15 dist-test-slave-2zvq krb5kdc[30468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440575, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:36:15.055758 30575 init.cc:377] Logged in from keytab as kudu/127.11.8.193@KRBTEST.COM (short username kudu)
I20260926 16:36:15.056617 30575 webserver.cc:460] Webserver started at http://127.11.8.193:41551/ using document root <none> and password file <none>
I20260926 16:36:15.057188 30575 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:15.057281 30575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:15.057513 30575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:15.059885 30575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-0/data/instance:
uuid: "b2e8332b7dea439280c12623b32de703"
format_stamp: "Formatted at 2026-09-26 16:36:15 on dist-test-slave-2zvq"
server_key: "dfa19f641a2cd81f55e46023cba9994a"
server_key_iv: "452ff2cad01c7ea9f72a0e8e442a77f8"
server_key_version: "encryptionkey@0"
I20260926 16:36:15.060472 30575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance:
uuid: "b2e8332b7dea439280c12623b32de703"
format_stamp: "Formatted at 2026-09-26 16:36:15 on dist-test-slave-2zvq"
server_key: "dfa19f641a2cd81f55e46023cba9994a"
server_key_iv: "452ff2cad01c7ea9f72a0e8e442a77f8"
server_key_version: "encryptionkey@0"
I20260926 16:36:15.064162 30575 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.001s
I20260926 16:36:15.066609 30590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:15.067832 30575 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260926 16:36:15.067953 30575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-0/wal
uuid: "b2e8332b7dea439280c12623b32de703"
format_stamp: "Formatted at 2026-09-26 16:36:15 on dist-test-slave-2zvq"
server_key: "dfa19f641a2cd81f55e46023cba9994a"
server_key_iv: "452ff2cad01c7ea9f72a0e8e442a77f8"
server_key_version: "encryptionkey@0"
I20260926 16:36:15.068049 30575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:15.079592 30575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:15.082738 30575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:15.082931 30575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:15.083480 30575 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:36:15.084240 30575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:36:15.084295 30575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:15.084362 30575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:36:15.084391 30575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:15.093117 30575 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.193:35321
I20260926 16:36:15.093187 30703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.193:35321 every 8 connection(s)
I20260926 16:36:15.093977 30575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
I20260926 16:36:15.097520 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 30575
I20260926 16:36:15.097651 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance
I20260926 16:36:15.097925 11299 external_mini_cluster.cc:1428] Setting key f58bb54e3006f2357fce4a09e183b360
Sep 26 16:36:15 dist-test-slave-2zvq krb5kdc[30468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440575, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.193@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.8.194@KRBTEST.COM; defaulting to no policy
I20260926 16:36:15.108122 30552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:15.095734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:37623 (local address 127.11.8.254:42265)
0926 16:36:15.095884 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:15.095888 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:15.096655 (+ 767us) server_negotiation.cc:407] Connection header received
0926 16:36:15.097997 (+ 1342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:15.098000 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:15.098058 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:15.098155 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:15.099537 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:15.100059 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:15.101179 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:15.101425 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:15.101534 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:15.104213 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:15.104219 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:15.104226 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:15.104261 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:15.105966 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:15.106511 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:15.106515 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:15.106517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:15.106603 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:15.106941 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:15.106945 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:15.106948 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:15.107190 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:15.107299 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:15.107763 (+ 464us) server_negotiation.cc:299] Negotiation successful
0926 16:36:15.107906 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 16:36:15.108826 30706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:15.096104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:42265 (local address 127.11.8.193:37623)
0926 16:36:15.096508 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:15.096548 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:15.097793 (+ 1245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:15.098395 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:15.098402 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:15.098808 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:36:15.099389 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:15.099404 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:15.100186 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:15.100190 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:15.101052 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:15.101061 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:15.101569 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:15.101574 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:15.101739 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:15.102385 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:36:15.102401 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:36:15.104051 (+ 1650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:36:15.106089 (+ 2038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:15.106093 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:15.106104 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:15.106415 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:15.106704 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:15.106706 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:15.106707 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:15.106835 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:15.107303 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:36:15.107308 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:36:15.107516 (+ 208us) client_negotiation.cc:769] Sending connection context
0926 16:36:15.107654 (+ 138us) client_negotiation.cc:240] Negotiation successful
0926 16:36:15.107950 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":99,"threads_started":1}
I20260926 16:36:15.110044 30704 heartbeater.cc:347] Connected to a master server at 127.11.8.254:42265
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.194@KRBTEST.COM" created.
I20260926 16:36:15.110302 30704 heartbeater.cc:464] Registering TS with master...
I20260926 16:36:15.110800 30704 heartbeater.cc:511] Master 127.11.8.254:42265 requested a full tablet report, sending...
I20260926 16:36:15.112226 30513 ts_manager.cc:194] Registered new tserver with Master: b2e8332b7dea439280c12623b32de703 (127.11.8.193:35321)
I20260926 16:36:15.113914 30513 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.193@KRBTEST.COM'} at 127.11.8.193:37623
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab.
Entry for principal kudu/127.11.8.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab.
I20260926 16:36:15.125363 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.194:0
--local_ip_for_outbound_sockets=127.11.8.194
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:42265
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:44499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:36:15.232314 30709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:15.232544 30709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:15.232584 30709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:15.236501 30709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:15.236548 30709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:15.236603 30709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.194
I20260926 16:36:15.241462 30709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:44499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.194.keytab
--principal=kudu/127.11.8.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:42265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.194
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:15.242309 30709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:15.243187 30709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:15.252305 30717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:15.252310 30714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:15.252348 30709 server_base.cc:1034] running on GCE node
W20260926 16:36:15.252315 30715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:15.252918 30709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:15.253518 30709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:15.254698 30709 hybrid_clock.cc:648] HybridClock initialized: now 1790440575254671 us; error 47 us; skew 500 ppm
Sep 26 16:36:15 dist-test-slave-2zvq krb5kdc[30468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440575, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:36:15.257203 30709 init.cc:377] Logged in from keytab as kudu/127.11.8.194@KRBTEST.COM (short username kudu)
I20260926 16:36:15.258042 30709 webserver.cc:460] Webserver started at http://127.11.8.194:43351/ using document root <none> and password file <none>
I20260926 16:36:15.258625 30709 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:15.258709 30709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:15.258929 30709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:15.260803 30709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-1/data/instance:
uuid: "5d44739c5eb2450a975dd9e3b093df42"
format_stamp: "Formatted at 2026-09-26 16:36:15 on dist-test-slave-2zvq"
server_key: "4594f335343ee1d0452c6f89d801c5ac"
server_key_iv: "63f3c6488135dfbe496d16e0d954bcee"
server_key_version: "encryptionkey@0"
I20260926 16:36:15.261333 30709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance:
uuid: "5d44739c5eb2450a975dd9e3b093df42"
format_stamp: "Formatted at 2026-09-26 16:36:15 on dist-test-slave-2zvq"
server_key: "4594f335343ee1d0452c6f89d801c5ac"
server_key_iv: "63f3c6488135dfbe496d16e0d954bcee"
server_key_version: "encryptionkey@0"
I20260926 16:36:15.264786 30709 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260926 16:36:15.267040 30724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:15.268139 30709 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:36:15.268268 30709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-1/wal
uuid: "5d44739c5eb2450a975dd9e3b093df42"
format_stamp: "Formatted at 2026-09-26 16:36:15 on dist-test-slave-2zvq"
server_key: "4594f335343ee1d0452c6f89d801c5ac"
server_key_iv: "63f3c6488135dfbe496d16e0d954bcee"
server_key_version: "encryptionkey@0"
I20260926 16:36:15.268380 30709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:15.293107 30709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:15.295854 30709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:15.296048 30709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:15.296607 30709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:36:15.297416 30709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:36:15.297487 30709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:15.297556 30709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:36:15.297600 30709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:15.306516 30709 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.194:39945
I20260926 16:36:15.306558 30837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.194:39945 every 8 connection(s)
I20260926 16:36:15.307361 30709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
I20260926 16:36:15.310926 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 30709
I20260926 16:36:15.311064 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance
I20260926 16:36:15.311355 11299 external_mini_cluster.cc:1428] Setting key 6fbed91f1e14cbfa6f0645a3f22bef86
Sep 26 16:36:15 dist-test-slave-2zvq krb5kdc[30468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440575, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.194@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.8.195@KRBTEST.COM; defaulting to no policy
I20260926 16:36:15.322021 30552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:15.309080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:48655 (local address 127.11.8.254:42265)
0926 16:36:15.309239 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:15.309243 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:15.309998 (+ 755us) server_negotiation.cc:407] Connection header received
0926 16:36:15.311550 (+ 1552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:15.311553 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:15.311599 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:15.311669 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:15.313006 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:15.313517 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:15.315004 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:15.315440 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:15.315575 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:15.318457 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:15.318460 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:15.318462 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:15.318484 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:15.319887 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:15.320506 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:15.320508 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:15.320509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:15.320578 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:15.320975 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:15.320978 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:15.320979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:15.321136 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:15.321206 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:15.321798 (+ 592us) server_negotiation.cc:299] Negotiation successful
0926 16:36:15.321904 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 16:36:15.322957 30840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:15.309443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:42265 (local address 127.11.8.194:48655)
0926 16:36:15.309852 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:15.309892 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:15.311316 (+ 1424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:15.311832 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:15.311839 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:15.312261 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:36:15.312861 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:15.312882 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:15.313645 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:15.313652 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:15.314875 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:15.314885 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:15.315596 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:15.315599 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:15.315732 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:15.316328 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:36:15.316363 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:36:15.318289 (+ 1926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:36:15.320008 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:15.320014 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:15.320029 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:15.320399 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:15.320683 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:15.320688 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:15.320693 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:15.320881 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:15.321253 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:36:15.321262 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:36:15.321578 (+ 316us) client_negotiation.cc:769] Sending connection context
0926 16:36:15.321733 (+ 155us) client_negotiation.cc:240] Negotiation successful
0926 16:36:15.322034 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":98,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.195@KRBTEST.COM" created.
I20260926 16:36:15.324258 30838 heartbeater.cc:347] Connected to a master server at 127.11.8.254:42265
I20260926 16:36:15.324513 30838 heartbeater.cc:464] Registering TS with master...
I20260926 16:36:15.325073 30838 heartbeater.cc:511] Master 127.11.8.254:42265 requested a full tablet report, sending...
I20260926 16:36:15.326222 30513 ts_manager.cc:194] Registered new tserver with Master: 5d44739c5eb2450a975dd9e3b093df42 (127.11.8.194:39945)
I20260926 16:36:15.326882 30513 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.194@KRBTEST.COM'} at 127.11.8.194:48655
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab.
Entry for principal kudu/127.11.8.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab.
I20260926 16:36:15.338954 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.195:0
--local_ip_for_outbound_sockets=127.11.8.195
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:42265
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:44499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:36:15.451092 30843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:15.451356 30843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:15.451429 30843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:15.455392 30843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:15.455499 30843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:15.455615 30843 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.195
I20260926 16:36:15.460717 30843 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:44499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.195.keytab
--principal=kudu/127.11.8.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:42265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.195
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:15.461679 30843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:15.462728 30843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:15.472121 30848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:15.472090 30849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:15.472090 30851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:15.472388 30843 server_base.cc:1034] running on GCE node
I20260926 16:36:15.472805 30843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:15.473464 30843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:15.474654 30843 hybrid_clock.cc:648] HybridClock initialized: now 1790440575474626 us; error 47 us; skew 500 ppm
Sep 26 16:36:15 dist-test-slave-2zvq krb5kdc[30468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440575, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:36:15.477676 30843 init.cc:377] Logged in from keytab as kudu/127.11.8.195@KRBTEST.COM (short username kudu)
I20260926 16:36:15.478551 30843 webserver.cc:460] Webserver started at http://127.11.8.195:36133/ using document root <none> and password file <none>
I20260926 16:36:15.479089 30843 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:15.479148 30843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:15.479377 30843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:15.481287 30843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-2/data/instance:
uuid: "794adbad2b4a40f2a76fe0e95a8cefb8"
format_stamp: "Formatted at 2026-09-26 16:36:15 on dist-test-slave-2zvq"
server_key: "ddfffdf8e4c35f59eca236c3fa7d891c"
server_key_iv: "20c57099ec2fb274f3a0c80a51e49768"
server_key_version: "encryptionkey@0"
I20260926 16:36:15.481832 30843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance:
uuid: "794adbad2b4a40f2a76fe0e95a8cefb8"
format_stamp: "Formatted at 2026-09-26 16:36:15 on dist-test-slave-2zvq"
server_key: "ddfffdf8e4c35f59eca236c3fa7d891c"
server_key_iv: "20c57099ec2fb274f3a0c80a51e49768"
server_key_version: "encryptionkey@0"
I20260926 16:36:15.485302 30843 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260926 16:36:15.487565 30858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:15.488695 30843 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 16:36:15.488822 30843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-2/wal
uuid: "794adbad2b4a40f2a76fe0e95a8cefb8"
format_stamp: "Formatted at 2026-09-26 16:36:15 on dist-test-slave-2zvq"
server_key: "ddfffdf8e4c35f59eca236c3fa7d891c"
server_key_iv: "20c57099ec2fb274f3a0c80a51e49768"
server_key_version: "encryptionkey@0"
I20260926 16:36:15.488934 30843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:15.499929 30843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:15.502568 30843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:15.502777 30843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:15.503316 30843 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:36:15.504129 30843 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:36:15.504197 30843 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:15.504268 30843 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:36:15.504310 30843 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:15.512414 30843 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.195:43327
I20260926 16:36:15.512480 30971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.195:43327 every 8 connection(s)
I20260926 16:36:15.513262 30843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
I20260926 16:36:15.514690 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 30843
I20260926 16:36:15.514837 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance
I20260926 16:36:15.515151 11299 external_mini_cluster.cc:1428] Setting key f7d5d7d2cee97573c6881ce9d057a336
Sep 26 16:36:15 dist-test-slave-2zvq krb5kdc[30468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440575, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.195@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:36:15.526342 30552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:15.515163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:53809 (local address 127.11.8.254:42265)
0926 16:36:15.515296 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:15.515301 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:15.516130 (+ 829us) server_negotiation.cc:407] Connection header received
0926 16:36:15.517603 (+ 1473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:15.517606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:15.517659 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:15.517742 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:15.518947 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:15.519491 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:15.520625 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:15.520868 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:15.520977 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:15.523307 (+ 2330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:15.523310 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:15.523312 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:15.523333 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:15.524572 (+ 1239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:15.525060 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:15.525063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:15.525064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:15.525136 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:15.525432 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:15.525434 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:15.525435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:15.525591 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:15.525663 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:15.526088 (+ 425us) server_negotiation.cc:299] Negotiation successful
0926 16:36:15.526177 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 16:36:15.527117 30974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:15.515545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:42265 (local address 127.11.8.195:53809)
0926 16:36:15.516003 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:15.516056 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:15.517415 (+ 1359us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:15.517897 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:15.517904 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:15.518229 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:36:15.518820 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:15.518835 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:15.519619 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:15.519623 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:15.520488 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:15.520497 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:15.521008 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:15.521012 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:15.521180 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:15.521754 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:36:15.521769 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:36:15.523160 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:36:15.524704 (+ 1544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:15.524709 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:15.524719 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:15.524967 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:15.525221 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:15.525223 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:15.525224 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:15.525353 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:15.525675 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:36:15.525680 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:36:15.525895 (+ 215us) client_negotiation.cc:769] Sending connection context
0926 16:36:15.526043 (+ 148us) client_negotiation.cc:240] Negotiation successful
0926 16:36:15.526348 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":96,"threads_started":1}
I20260926 16:36:15.528298 30972 heartbeater.cc:347] Connected to a master server at 127.11.8.254:42265
I20260926 16:36:15.528538 30972 heartbeater.cc:464] Registering TS with master...
I20260926 16:36:15.529040 30972 heartbeater.cc:511] Master 127.11.8.254:42265 requested a full tablet report, sending...
I20260926 16:36:15.529980 30512 ts_manager.cc:194] Registered new tserver with Master: 794adbad2b4a40f2a76fe0e95a8cefb8 (127.11.8.195:43327)
I20260926 16:36:15.530615 30512 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.195@KRBTEST.COM'} at 127.11.8.195:53809
I20260926 16:36:15.540491 11299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 16:36:15.549583 30552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:15.542385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37142 (local address 127.11.8.254:42265)
0926 16:36:15.542511 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:15.542515 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:15.542684 (+ 169us) server_negotiation.cc:407] Connection header received
0926 16:36:15.542796 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:15.542798 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:15.542834 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:15.542897 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:15.543508 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:15.543983 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:15.544787 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:15.545044 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:15.545118 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:15.546653 (+ 1535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:15.546657 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:15.546658 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:15.546679 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:15.547977 (+ 1298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:15.548391 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:15.548395 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:15.548397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:15.548474 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:15.548745 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:15.548748 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:15.548750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:15.548950 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:15.549084 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:15.549309 (+ 225us) server_negotiation.cc:299] Negotiation successful
0926 16:36:15.549422 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 16:36:15.552377 30512 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37142:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:36:15.555277 30512 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:36:15.569205 30983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:15.563306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:39945 (local address 127.0.0.1:53308)
0926 16:36:15.563817 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:15.563869 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:15.563985 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:15.564912 (+ 927us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:15.564918 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:15.564943 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:36:15.565089 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:15.565097 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:15.566555 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:15.566558 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:15.567817 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:15.567826 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:15.568668 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:15.568671 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:15.568793 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:15.568845 (+ 52us) client_negotiation.cc:769] Sending connection context
0926 16:36:15.568950 (+ 105us) client_negotiation.cc:240] Negotiation successful
0926 16:36:15.569063 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":92,"threads_started":1}
I20260926 16:36:15.569242 30986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:15.563999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53308 (local address 127.11.8.194:39945)
0926 16:36:15.564334 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:15.564347 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:15.564373 (+ 26us) server_negotiation.cc:407] Connection header received
0926 16:36:15.564448 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:15.564452 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:15.564660 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:15.564925 (+ 265us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:36:15.565260 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:15.566411 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:15.567936 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:15.568554 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:15.568662 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:15.568809 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:15.568997 (+ 188us) server_negotiation.cc:299] Negotiation successful
0926 16:36:15.569098 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":106,"threads_started":1}
I20260926 16:36:15.569902 30984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:15.563503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:43327 (local address 127.0.0.1:47948)
0926 16:36:15.564781 (+ 1278us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:15.564796 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:15.564918 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:15.565612 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:15.565615 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:15.565631 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:36:15.565725 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:15.565731 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:15.567261 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:15.567265 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:15.568328 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:15.568338 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:15.569503 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:15.569506 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:15.569613 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:15.569644 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 16:36:15.569704 (+ 60us) client_negotiation.cc:240] Negotiation successful
0926 16:36:15.569784 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1213,"thread_start_us":89,"threads_started":1}
I20260926 16:36:15.570608 30987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:15.564421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47948 (local address 127.11.8.195:43327)
0926 16:36:15.564732 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:15.564746 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:15.565223 (+ 477us) server_negotiation.cc:407] Connection header received
0926 16:36:15.565307 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:15.565313 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:15.565482 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:15.565634 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:36:15.565817 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:15.567127 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:15.568479 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:15.569300 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:15.569404 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:15.570387 (+ 983us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:15.570439 (+ 52us) server_negotiation.cc:299] Negotiation successful
0926 16:36:15.570507 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":94,"threads_started":1}
I20260926 16:36:15.570613 30985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:15.564017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:35321 (local address 127.0.0.1:39328)
0926 16:36:15.565194 (+ 1177us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:15.565209 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:15.565308 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:15.565925 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:15.565927 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:15.565943 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:36:15.566045 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:15.566051 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:15.567654 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:15.567657 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:15.568723 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:15.568731 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:15.570120 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:15.570124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:15.570216 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:15.570239 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 16:36:15.570418 (+ 179us) client_negotiation.cc:240] Negotiation successful
0926 16:36:15.570494 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1101,"thread_start_us":94,"threads_started":1}
I20260926 16:36:15.571357 30988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:15.564470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39328 (local address 127.11.8.193:35321)
0926 16:36:15.564872 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:15.564883 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:15.565332 (+ 449us) server_negotiation.cc:407] Connection header received
0926 16:36:15.565417 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:15.565422 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:15.565609 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:15.565777 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:36:15.566167 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:15.567521 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:15.568863 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:15.569904 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:15.570908 (+ 1004us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:15.571037 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:15.571134 (+ 97us) server_negotiation.cc:299] Negotiation successful
0926 16:36:15.571221 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":116,"threads_started":1}
I20260926 16:36:15.572808 30772 tablet_service.cc:1467] Processing CreateTablet for tablet a06698612c824347b91c7bd5b0b4b390 (DEFAULT_TABLE table=test-table [id=a482206469854cd2a32c54aec4c082d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:36:15.573419 30906 tablet_service.cc:1467] Processing CreateTablet for tablet a06698612c824347b91c7bd5b0b4b390 (DEFAULT_TABLE table=test-table [id=a482206469854cd2a32c54aec4c082d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:36:15.573951 30772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a06698612c824347b91c7bd5b0b4b390. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:15.574409 30906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a06698612c824347b91c7bd5b0b4b390. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:15.574112 30638 tablet_service.cc:1467] Processing CreateTablet for tablet a06698612c824347b91c7bd5b0b4b390 (DEFAULT_TABLE table=test-table [id=a482206469854cd2a32c54aec4c082d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:36:15.575038 30638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a06698612c824347b91c7bd5b0b4b390. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:15.580279 30989 tablet_bootstrap.cc:492] T a06698612c824347b91c7bd5b0b4b390 P 794adbad2b4a40f2a76fe0e95a8cefb8: Bootstrap starting.
I20260926 16:36:15.581959 30991 tablet_bootstrap.cc:492] T a06698612c824347b91c7bd5b0b4b390 P b2e8332b7dea439280c12623b32de703: Bootstrap starting.
I20260926 16:36:15.582530 30989 tablet_bootstrap.cc:654] T a06698612c824347b91c7bd5b0b4b390 P 794adbad2b4a40f2a76fe0e95a8cefb8: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:15.582585 30990 tablet_bootstrap.cc:492] T a06698612c824347b91c7bd5b0b4b390 P 5d44739c5eb2450a975dd9e3b093df42: Bootstrap starting.
I20260926 16:36:15.583554 30989 log.cc:824] T a06698612c824347b91c7bd5b0b4b390 P 794adbad2b4a40f2a76fe0e95a8cefb8: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:15.584071 30991 tablet_bootstrap.cc:654] T a06698612c824347b91c7bd5b0b4b390 P b2e8332b7dea439280c12623b32de703: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:15.585005 30990 tablet_bootstrap.cc:654] T a06698612c824347b91c7bd5b0b4b390 P 5d44739c5eb2450a975dd9e3b093df42: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:15.585299 30991 log.cc:824] T a06698612c824347b91c7bd5b0b4b390 P b2e8332b7dea439280c12623b32de703: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:15.585496 30989 tablet_bootstrap.cc:492] T a06698612c824347b91c7bd5b0b4b390 P 794adbad2b4a40f2a76fe0e95a8cefb8: No bootstrap required, opened a new log
I20260926 16:36:15.585736 30989 ts_tablet_manager.cc:1397] T a06698612c824347b91c7bd5b0b4b390 P 794adbad2b4a40f2a76fe0e95a8cefb8: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 16:36:15.585845 30990 log.cc:824] T a06698612c824347b91c7bd5b0b4b390 P 5d44739c5eb2450a975dd9e3b093df42: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:15.587215 30991 tablet_bootstrap.cc:492] T a06698612c824347b91c7bd5b0b4b390 P b2e8332b7dea439280c12623b32de703: No bootstrap required, opened a new log
I20260926 16:36:15.587442 30991 ts_tablet_manager.cc:1397] T a06698612c824347b91c7bd5b0b4b390 P b2e8332b7dea439280c12623b32de703: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260926 16:36:15.587661 30990 tablet_bootstrap.cc:492] T a06698612c824347b91c7bd5b0b4b390 P 5d44739c5eb2450a975dd9e3b093df42: No bootstrap required, opened a new log
I20260926 16:36:15.587831 30990 ts_tablet_manager.cc:1397] T a06698612c824347b91c7bd5b0b4b390 P 5d44739c5eb2450a975dd9e3b093df42: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260926 16:36:15.588935 30989 raft_consensus.cc:348] T a06698612c824347b91c7bd5b0b4b390 P 794adbad2b4a40f2a76fe0e95a8cefb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d44739c5eb2450a975dd9e3b093df42" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 39945 } } peers { permanent_uuid: "794adbad2b4a40f2a76fe0e95a8cefb8" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 43327 } } peers { permanent_uuid: "b2e8332b7dea439280c12623b32de703" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 35321 } }
I20260926 16:36:15.589174 30989 raft_consensus.cc:374] T a06698612c824347b91c7bd5b0b4b390 P 794adbad2b4a40f2a76fe0e95a8cefb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:36:15.589246 30989 raft_consensus.cc:729] T a06698612c824347b91c7bd5b0b4b390 P 794adbad2b4a40f2a76fe0e95a8cefb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 794adbad2b4a40f2a76fe0e95a8cefb8, State: Initialized, Role: FOLLOWER
I20260926 16:36:15.589684 30989 consensus_queue.cc:260] T a06698612c824347b91c7bd5b0b4b390 P 794adbad2b4a40f2a76fe0e95a8cefb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d44739c5eb2450a975dd9e3b093df42" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 39945 } } peers { permanent_uuid: "794adbad2b4a40f2a76fe0e95a8cefb8" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 43327 } } peers { permanent_uuid: "b2e8332b7dea439280c12623b32de703" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 35321 } }
I20260926 16:36:15.590364 30972 heartbeater.cc:503] Master 127.11.8.254:42265 was elected leader, sending a full tablet report...
I20260926 16:36:15.591298 30991 raft_consensus.cc:348] T a06698612c824347b91c7bd5b0b4b390 P b2e8332b7dea439280c12623b32de703 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d44739c5eb2450a975dd9e3b093df42" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 39945 } } peers { permanent_uuid: "794adbad2b4a40f2a76fe0e95a8cefb8" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 43327 } } peers { permanent_uuid: "b2e8332b7dea439280c12623b32de703" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 35321 } }
I20260926 16:36:15.591405 30990 raft_consensus.cc:348] T a06698612c824347b91c7bd5b0b4b390 P 5d44739c5eb2450a975dd9e3b093df42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d44739c5eb2450a975dd9e3b093df42" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 39945 } } peers { permanent_uuid: "794adbad2b4a40f2a76fe0e95a8cefb8" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 43327 } } peers { permanent_uuid: "b2e8332b7dea439280c12623b32de703" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 35321 } }
I20260926 16:36:15.591603 30991 raft_consensus.cc:374] T a06698612c824347b91c7bd5b0b4b390 P b2e8332b7dea439280c12623b32de703 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:36:15.591610 30989 ts_tablet_manager.cc:1428] T a06698612c824347b91c7bd5b0b4b390 P 794adbad2b4a40f2a76fe0e95a8cefb8: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260926 16:36:15.591658 30990 raft_consensus.cc:374] T a06698612c824347b91c7bd5b0b4b390 P 5d44739c5eb2450a975dd9e3b093df42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:36:15.591696 30991 raft_consensus.cc:729] T a06698612c824347b91c7bd5b0b4b390 P b2e8332b7dea439280c12623b32de703 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e8332b7dea439280c12623b32de703, State: Initialized, Role: FOLLOWER
I20260926 16:36:15.591778 30990 raft_consensus.cc:729] T a06698612c824347b91c7bd5b0b4b390 P 5d44739c5eb2450a975dd9e3b093df42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d44739c5eb2450a975dd9e3b093df42, State: Initialized, Role: FOLLOWER
I20260926 16:36:15.592247 30990 consensus_queue.cc:260] T a06698612c824347b91c7bd5b0b4b390 P 5d44739c5eb2450a975dd9e3b093df42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d44739c5eb2450a975dd9e3b093df42" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 39945 } } peers { permanent_uuid: "794adbad2b4a40f2a76fe0e95a8cefb8" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 43327 } } peers { permanent_uuid: "b2e8332b7dea439280c12623b32de703" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 35321 } }
I20260926 16:36:15.592244 30991 consensus_queue.cc:260] T a06698612c824347b91c7bd5b0b4b390 P b2e8332b7dea439280c12623b32de703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d44739c5eb2450a975dd9e3b093df42" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 39945 } } peers { permanent_uuid: "794adbad2b4a40f2a76fe0e95a8cefb8" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 43327 } } peers { permanent_uuid: "b2e8332b7dea439280c12623b32de703" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 35321 } }
I20260926 16:36:15.593168 30990 ts_tablet_manager.cc:1428] T a06698612c824347b91c7bd5b0b4b390 P 5d44739c5eb2450a975dd9e3b093df42: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260926 16:36:15.593168 30991 ts_tablet_manager.cc:1428] T a06698612c824347b91c7bd5b0b4b390 P b2e8332b7dea439280c12623b32de703: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260926 16:36:15.593602 30704 heartbeater.cc:503] Master 127.11.8.254:42265 was elected leader, sending a full tablet report...
I20260926 16:36:15.594417 30838 heartbeater.cc:503] Master 127.11.8.254:42265 was elected leader, sending a full tablet report...
W20260926 16:36:15.595121 30705 tablet.cc:2369] T a06698612c824347b91c7bd5b0b4b390 P b2e8332b7dea439280c12623b32de703: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:36:15.738775 30997 raft_consensus.cc:482] T a06698612c824347b91c7bd5b0b4b390 P 5d44739c5eb2450a975dd9e3b093df42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:36:15.739017 30997 raft_consensus.cc:504] T a06698612c824347b91c7bd5b0b4b390 P 5d44739c5eb2450a975dd9e3b093df42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d44739c5eb2450a975dd9e3b093df42" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 39945 } } peers { permanent_uuid: "794adbad2b4a40f2a76fe0e95a8cefb8" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 43327 } } peers { permanent_uuid: "b2e8332b7dea439280c12623b32de703" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 35321 } }
I20260926 16:36:15.740263 30997 leader_election.cc:288] T a06698612c824347b91c7bd5b0b4b390 P 5d44739c5eb2450a975dd9e3b093df42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 794adbad2b4a40f2a76fe0e95a8cefb8 (127.11.8.195:43327), b2e8332b7dea439280c12623b32de703 (127.11.8.193:35321)
I20260926 16:36:15.744181 30840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:15.740498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:43327 (local address 127.11.8.194:51187)
0926 16:36:15.740696 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:15.740712 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:15.740840 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:15.741136 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:15.741139 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:15.741157 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:36:15.741299 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:15.741306 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:15.742258 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:15.742293 (+ 35us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:15.743079 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:15.743089 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:15.743724 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:15.743728 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:15.743830 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:15.743861 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 16:36:15.743949 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 16:36:15.744023 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260926 16:36:15.744180 30987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:15.740608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:51187 (local address 127.11.8.195:43327)
0926 16:36:15.740744 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:15.740750 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:15.740768 (+ 18us) server_negotiation.cc:407] Connection header received
0926 16:36:15.740977 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:15.740981 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:15.741036 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:15.741120 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:36:15.741616 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:15.742146 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:15.743202 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:15.743606 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:15.743733 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:15.743786 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:15.743964 (+ 178us) server_negotiation.cc:299] Negotiation successful
0926 16:36:15.744031 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 16:36:15.744411 30988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:15.740924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:38911 (local address 127.11.8.193:35321)
0926 16:36:15.741061 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:15.741067 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:15.741254 (+ 187us) server_negotiation.cc:407] Connection header received
0926 16:36:15.741422 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:15.741425 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:15.741465 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:15.741526 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:36:15.741825 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:15.742466 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:15.743499 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:15.743893 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:15.744015 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:15.744064 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:15.744226 (+ 162us) server_negotiation.cc:299] Negotiation successful
0926 16:36:15.744294 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 16:36:15.744426 30998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:15.740797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:35321 (local address 127.11.8.194:38911)
0926 16:36:15.741170 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:15.741183 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:15.741307 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:15.741549 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:15.741551 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:15.741564 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:36:15.741711 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:15.741723 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:15.742567 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:15.742573 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:15.743375 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:15.743384 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:15.744008 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:15.744012 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:15.744111 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:15.744129 (+ 18us) client_negotiation.cc:769] Sending connection context
0926 16:36:15.744210 (+ 81us) client_negotiation.cc:240] Negotiation successful
0926 16:36:15.744288 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":134,"threads_started":1}
I20260926 16:36:15.745009 30658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a06698612c824347b91c7bd5b0b4b390" candidate_uuid: "5d44739c5eb2450a975dd9e3b093df42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e8332b7dea439280c12623b32de703" is_pre_election: true
I20260926 16:36:15.745005 30926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a06698612c824347b91c7bd5b0b4b390" candidate_uuid: "5d44739c5eb2450a975dd9e3b093df42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "794adbad2b4a40f2a76fe0e95a8cefb8" is_pre_election: true
I20260926 16:36:15.745316 30926 raft_consensus.cc:2454] T a06698612c824347b91c7bd5b0b4b390 P 794adbad2b4a40f2a76fe0e95a8cefb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d44739c5eb2450a975dd9e3b093df42 in term 0.
I20260926 16:36:15.745316 30658 raft_consensus.cc:2454] T a06698612c824347b91c7bd5b0b4b390 P b2e8332b7dea439280c12623b32de703 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d44739c5eb2450a975dd9e3b093df42 in term 0.
I20260926 16:36:15.745805 30726 leader_election.cc:302] T a06698612c824347b91c7bd5b0b4b390 P 5d44739c5eb2450a975dd9e3b093df42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d44739c5eb2450a975dd9e3b093df42, 794adbad2b4a40f2a76fe0e95a8cefb8; no voters:
I20260926 16:36:15.746047 30997 raft_consensus.cc:2781] T a06698612c824347b91c7bd5b0b4b390 P 5d44739c5eb2450a975dd9e3b093df42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:36:15.746131 30997 raft_consensus.cc:482] T a06698612c824347b91c7bd5b0b4b390 P 5d44739c5eb2450a975dd9e3b093df42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:36:15.746171 30997 raft_consensus.cc:3037] T a06698612c824347b91c7bd5b0b4b390 P 5d44739c5eb2450a975dd9e3b093df42 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:15.747213 30997 raft_consensus.cc:504] T a06698612c824347b91c7bd5b0b4b390 P 5d44739c5eb2450a975dd9e3b093df42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d44739c5eb2450a975dd9e3b093df42" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 39945 } } peers { permanent_uuid: "794adbad2b4a40f2a76fe0e95a8cefb8" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 43327 } } peers { permanent_uuid: "b2e8332b7dea439280c12623b32de703" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 35321 } }
I20260926 16:36:15.747579 30997 leader_election.cc:288] T a06698612c824347b91c7bd5b0b4b390 P 5d44739c5eb2450a975dd9e3b093df42 [CANDIDATE]: Term 1 election: Requested vote from peers 794adbad2b4a40f2a76fe0e95a8cefb8 (127.11.8.195:43327), b2e8332b7dea439280c12623b32de703 (127.11.8.193:35321)
I20260926 16:36:15.747944 30926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a06698612c824347b91c7bd5b0b4b390" candidate_uuid: "5d44739c5eb2450a975dd9e3b093df42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "794adbad2b4a40f2a76fe0e95a8cefb8"
I20260926 16:36:15.747944 30658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a06698612c824347b91c7bd5b0b4b390" candidate_uuid: "5d44739c5eb2450a975dd9e3b093df42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e8332b7dea439280c12623b32de703"
I20260926 16:36:15.748067 30926 raft_consensus.cc:3037] T a06698612c824347b91c7bd5b0b4b390 P 794adbad2b4a40f2a76fe0e95a8cefb8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:15.748067 30658 raft_consensus.cc:3037] T a06698612c824347b91c7bd5b0b4b390 P b2e8332b7dea439280c12623b32de703 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:15.749290 30926 raft_consensus.cc:2454] T a06698612c824347b91c7bd5b0b4b390 P 794adbad2b4a40f2a76fe0e95a8cefb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d44739c5eb2450a975dd9e3b093df42 in term 1.
I20260926 16:36:15.749290 30658 raft_consensus.cc:2454] T a06698612c824347b91c7bd5b0b4b390 P b2e8332b7dea439280c12623b32de703 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d44739c5eb2450a975dd9e3b093df42 in term 1.
I20260926 16:36:15.749585 30726 leader_election.cc:302] T a06698612c824347b91c7bd5b0b4b390 P 5d44739c5eb2450a975dd9e3b093df42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d44739c5eb2450a975dd9e3b093df42, b2e8332b7dea439280c12623b32de703; no voters:
I20260926 16:36:15.749765 30997 raft_consensus.cc:2781] T a06698612c824347b91c7bd5b0b4b390 P 5d44739c5eb2450a975dd9e3b093df42 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:36:15.750036 30997 raft_consensus.cc:686] T a06698612c824347b91c7bd5b0b4b390 P 5d44739c5eb2450a975dd9e3b093df42 [term 1 LEADER]: Becoming Leader. State: Replica: 5d44739c5eb2450a975dd9e3b093df42, State: Running, Role: LEADER
I20260926 16:36:15.750423 30997 consensus_queue.cc:237] T a06698612c824347b91c7bd5b0b4b390 P 5d44739c5eb2450a975dd9e3b093df42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d44739c5eb2450a975dd9e3b093df42" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 39945 } } peers { permanent_uuid: "794adbad2b4a40f2a76fe0e95a8cefb8" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 43327 } } peers { permanent_uuid: "b2e8332b7dea439280c12623b32de703" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 35321 } }
I20260926 16:36:15.753532 30512 catalog_manager.cc:5495] T a06698612c824347b91c7bd5b0b4b390 P 5d44739c5eb2450a975dd9e3b093df42 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d44739c5eb2450a975dd9e3b093df42 (127.11.8.194). New cstate: current_term: 1 leader_uuid: "5d44739c5eb2450a975dd9e3b093df42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d44739c5eb2450a975dd9e3b093df42" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 39945 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "794adbad2b4a40f2a76fe0e95a8cefb8" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e8332b7dea439280c12623b32de703" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 35321 } health_report { overall_health: UNKNOWN } } }
W20260926 16:36:15.764189 30973 tablet.cc:2369] T a06698612c824347b91c7bd5b0b4b390 P 794adbad2b4a40f2a76fe0e95a8cefb8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:36:15.803537 30986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:15.799977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53316 (local address 127.11.8.194:39945)
0926 16:36:15.800118 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:15.800122 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:15.800137 (+ 15us) server_negotiation.cc:407] Connection header received
0926 16:36:15.800269 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:15.800272 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:15.800325 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:15.800439 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:36:15.800701 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:15.801201 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:15.802042 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:15.802305 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:15.802413 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:15.802719 (+ 306us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:36:15.803160 (+ 441us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:36:15.803286 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:15.803334 (+ 48us) server_negotiation.cc:299] Negotiation successful
0926 16:36:15.803385 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260926 16:36:15.808461 30839 tablet.cc:2369] T a06698612c824347b91c7bd5b0b4b390 P 5d44739c5eb2450a975dd9e3b093df42: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:36:15.809319 30926 raft_consensus.cc:1261] T a06698612c824347b91c7bd5b0b4b390 P 794adbad2b4a40f2a76fe0e95a8cefb8 [term 1 FOLLOWER]: Refusing update from remote peer 5d44739c5eb2450a975dd9e3b093df42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:36:15.809321 30658 raft_consensus.cc:1261] T a06698612c824347b91c7bd5b0b4b390 P b2e8332b7dea439280c12623b32de703 [term 1 FOLLOWER]: Refusing update from remote peer 5d44739c5eb2450a975dd9e3b093df42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:36:15.809986 30997 consensus_queue.cc:1035] T a06698612c824347b91c7bd5b0b4b390 P 5d44739c5eb2450a975dd9e3b093df42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e8332b7dea439280c12623b32de703" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 35321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:36:15.810189 30999 consensus_queue.cc:1035] T a06698612c824347b91c7bd5b0b4b390 P 5d44739c5eb2450a975dd9e3b093df42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "794adbad2b4a40f2a76fe0e95a8cefb8" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 43327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:36:15.817508 31004 mvcc.cc:205] Tried to move back new op lower bound from 7333644598507376640 to 7333644598276366336. Current Snapshot: MvccSnapshot[applied={T|T < 7333644598507376640}]
I20260926 16:36:15.818804 31003 mvcc.cc:205] Tried to move back new op lower bound from 7333644598507376640 to 7333644598276366336. Current Snapshot: MvccSnapshot[applied={T|T < 7333644598507376640}]
I20260926 16:36:15.824620 30513 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37142:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:36:15.824775 30513 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37142:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:36:15.827193 30513 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d602f22fa25f43e48a324b47992b9204: Sending DeleteTablet for 3 replicas of tablet a06698612c824347b91c7bd5b0b4b390
I20260926 16:36:15.827883 30772 tablet_service.cc:1514] Processing DeleteTablet for tablet a06698612c824347b91c7bd5b0b4b390 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:36:15 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:53308
I20260926 16:36:15.827960 30906 tablet_service.cc:1514] Processing DeleteTablet for tablet a06698612c824347b91c7bd5b0b4b390 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:36:15 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:47948
I20260926 16:36:15.828097 30638 tablet_service.cc:1514] Processing DeleteTablet for tablet a06698612c824347b91c7bd5b0b4b390 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:36:15 UTC) from {username='kudu', principal='kudu/127.11.8.254@KRBTEST.COM'} at 127.0.0.1:39328
I20260926 16:36:15.828866 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 30575
I20260926 16:36:15.837288 31012 tablet_replica.cc:330] T a06698612c824347b91c7bd5b0b4b390 P 794adbad2b4a40f2a76fe0e95a8cefb8: stopping tablet replica
I20260926 16:36:15.837627 31012 raft_consensus.cc:2229] T a06698612c824347b91c7bd5b0b4b390 P 794adbad2b4a40f2a76fe0e95a8cefb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:36:15.837924 31012 raft_consensus.cc:2258] T a06698612c824347b91c7bd5b0b4b390 P 794adbad2b4a40f2a76fe0e95a8cefb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:36:15.838228 31011 tablet_replica.cc:330] T a06698612c824347b91c7bd5b0b4b390 P 5d44739c5eb2450a975dd9e3b093df42: stopping tablet replica
I20260926 16:36:15.838511 31011 raft_consensus.cc:2229] T a06698612c824347b91c7bd5b0b4b390 P 5d44739c5eb2450a975dd9e3b093df42 [term 1 LEADER]: Raft consensus shutting down.
I20260926 16:36:15.838855 31011 raft_consensus.cc:2258] T a06698612c824347b91c7bd5b0b4b390 P 5d44739c5eb2450a975dd9e3b093df42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:36:15.840250 31011 ts_tablet_manager.cc:1905] T a06698612c824347b91c7bd5b0b4b390 P 5d44739c5eb2450a975dd9e3b093df42: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 16:36:15.840363 30498 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.8.193:35321 (error 108)
I20260926 16:36:15.841601 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 30709
I20260926 16:36:15.841683 30985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:15.841394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:35321 (local address 127.0.0.1:39336)
0926 16:36:15.841510 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:15.841586 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.193:35321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260926 16:36:15.843364 31012 ts_tablet_manager.cc:1905] T a06698612c824347b91c7bd5b0b4b390 P 794adbad2b4a40f2a76fe0e95a8cefb8: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 16:36:15.843492 30498 catalog_manager.cc:4584] TS b2e8332b7dea439280c12623b32de703 (127.11.8.193:35321): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a06698612c824347b91c7bd5b0b4b390: Network error: Client connection negotiation failed: client connection to 127.11.8.193:35321: connect: Connection refused (error 111)
I20260926 16:36:15.846709 31012 ts_tablet_manager.cc:1918] T a06698612c824347b91c7bd5b0b4b390 P 794adbad2b4a40f2a76fe0e95a8cefb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:36:15.846809 31012 log.cc:1196] T a06698612c824347b91c7bd5b0b4b390 P 794adbad2b4a40f2a76fe0e95a8cefb8: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790440324823152-11299-0/minicluster-data/ts-2/wal/wals/a06698612c824347b91c7bd5b0b4b390
I20260926 16:36:15.847154 31012 ts_tablet_manager.cc:1939] T a06698612c824347b91c7bd5b0b4b390 P 794adbad2b4a40f2a76fe0e95a8cefb8: Deleting consensus metadata
I20260926 16:36:15.848392 30498 catalog_manager.cc:4841] TS 794adbad2b4a40f2a76fe0e95a8cefb8 (127.11.8.195:43327): tablet a06698612c824347b91c7bd5b0b4b390 (table test-table [id=a482206469854cd2a32c54aec4c082d1]) successfully deleted
I20260926 16:36:15.849061 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 30843
I20260926 16:36:15.849735 30985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:15.849202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:39945 (local address 127.0.0.1:53320)
0926 16:36:15.849622 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:15.849676 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.194:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345}
I20260926 16:36:15.854673 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 30481
2026-09-26T16:36:15Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2353 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-26T16:36:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:36:15Z Disabled control of system clock
I20260926 16:36:15.884186 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:38357
--webserver_interface=127.11.8.254
--webserver_port=0
--builtin_ntp_servers=127.11.8.212:42743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:38357
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 16:36:15.990077 31018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:15.990371 31018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:15.990454 31018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:15.994421 31018 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:36:15.994494 31018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:15.994521 31018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:15.994560 31018 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:36:15.994593 31018 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:36:15.999229 31018 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:42743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:38357
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:38357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:16.000244 31018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:16.001570 31018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:16.009760 31026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:16.009740 31023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:16.009795 31018 server_base.cc:1034] running on GCE node
W20260926 16:36:16.009742 31024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:16.010596 31018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:16.011580 31018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:16.012741 31018 hybrid_clock.cc:648] HybridClock initialized: now 1790440576012724 us; error 29 us; skew 500 ppm
I20260926 16:36:16.014480 31018 webserver.cc:460] Webserver started at http://127.11.8.254:46511/ using document root <none> and password file <none>
I20260926 16:36:16.014983 31018 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:16.015041 31018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:16.015194 31018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:16.017086 31018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "0315486fe3ed4241bff355afa6d87e16"
format_stamp: "Formatted at 2026-09-26 16:36:16 on dist-test-slave-2zvq"
server_key: "b19ce505c3744025add5b393261214ea"
server_key_iv: "ea61d5dc07bf33f35f1b8c70c14739c6"
server_key_version: "encryptionkey@0"
I20260926 16:36:16.017590 31018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "0315486fe3ed4241bff355afa6d87e16"
format_stamp: "Formatted at 2026-09-26 16:36:16 on dist-test-slave-2zvq"
server_key: "b19ce505c3744025add5b393261214ea"
server_key_iv: "ea61d5dc07bf33f35f1b8c70c14739c6"
server_key_version: "encryptionkey@0"
I20260926 16:36:16.021130 31018 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260926 16:36:16.023488 31032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:16.024667 31018 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 16:36:16.024787 31018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "0315486fe3ed4241bff355afa6d87e16"
format_stamp: "Formatted at 2026-09-26 16:36:16 on dist-test-slave-2zvq"
server_key: "b19ce505c3744025add5b393261214ea"
server_key_iv: "ea61d5dc07bf33f35f1b8c70c14739c6"
server_key_version: "encryptionkey@0"
I20260926 16:36:16.024893 31018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:16.036715 31018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:16.037312 31018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:16.037513 31018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:16.044433 31084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:38357 every 8 connection(s)
I20260926 16:36:16.044432 31018 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:38357
I20260926 16:36:16.045352 31018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:36:16.048261 31085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:16.049441 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 31018
I20260926 16:36:16.049563 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:36:16.049865 11299 external_mini_cluster.cc:1428] Setting key 9bb6cf2fe95e6a0f87ff99b90c383ec0
I20260926 16:36:16.054136 31085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0315486fe3ed4241bff355afa6d87e16: Bootstrap starting.
I20260926 16:36:16.056262 31085 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0315486fe3ed4241bff355afa6d87e16: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:16.057080 31085 log.cc:824] T 00000000000000000000000000000000 P 0315486fe3ed4241bff355afa6d87e16: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:16.058780 31088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:16.051292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60298 (local address 127.11.8.254:38357)
0926 16:36:16.051903 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:16.051917 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:16.051952 (+ 35us) server_negotiation.cc:407] Connection header received
0926 16:36:16.052565 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:16.052581 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:16.052958 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:16.053282 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:16.053697 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:16.054706 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:16.055581 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:16.056044 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:16.056215 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:16.057044 (+ 829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:16.057049 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:16.057061 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:36:16.057486 (+ 425us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:16.057538 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:36:16.057558 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:16.057666 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:16.057814 (+ 148us) server_negotiation.cc:299] Negotiation successful
0926 16:36:16.058060 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":112,"threads_started":1}
I20260926 16:36:16.058984 31085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0315486fe3ed4241bff355afa6d87e16: No bootstrap required, opened a new log
I20260926 16:36:16.062634 31085 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0315486fe3ed4241bff355afa6d87e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0315486fe3ed4241bff355afa6d87e16" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 38357 } }
I20260926 16:36:16.062870 31085 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0315486fe3ed4241bff355afa6d87e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:36:16.062952 31085 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0315486fe3ed4241bff355afa6d87e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0315486fe3ed4241bff355afa6d87e16, State: Initialized, Role: FOLLOWER
I20260926 16:36:16.063380 31085 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0315486fe3ed4241bff355afa6d87e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0315486fe3ed4241bff355afa6d87e16" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 38357 } }
I20260926 16:36:16.063520 31085 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0315486fe3ed4241bff355afa6d87e16 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:36:16.063606 31085 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0315486fe3ed4241bff355afa6d87e16 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:36:16.063699 31085 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0315486fe3ed4241bff355afa6d87e16 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:16.064507 31085 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0315486fe3ed4241bff355afa6d87e16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0315486fe3ed4241bff355afa6d87e16" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 38357 } }
I20260926 16:36:16.064822 31085 leader_election.cc:302] T 00000000000000000000000000000000 P 0315486fe3ed4241bff355afa6d87e16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0315486fe3ed4241bff355afa6d87e16; no voters:
I20260926 16:36:16.065133 31085 leader_election.cc:288] T 00000000000000000000000000000000 P 0315486fe3ed4241bff355afa6d87e16 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:36:16.065253 31090 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0315486fe3ed4241bff355afa6d87e16 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:36:16.065528 31090 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0315486fe3ed4241bff355afa6d87e16 [term 1 LEADER]: Becoming Leader. State: Replica: 0315486fe3ed4241bff355afa6d87e16, State: Running, Role: LEADER
I20260926 16:36:16.065845 31090 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0315486fe3ed4241bff355afa6d87e16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0315486fe3ed4241bff355afa6d87e16" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 38357 } }
I20260926 16:36:16.066331 31085 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0315486fe3ed4241bff355afa6d87e16 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:36:16.067488 31092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0315486fe3ed4241bff355afa6d87e16 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0315486fe3ed4241bff355afa6d87e16. Latest consensus state: current_term: 1 leader_uuid: "0315486fe3ed4241bff355afa6d87e16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0315486fe3ed4241bff355afa6d87e16" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 38357 } } }
I20260926 16:36:16.067513 31091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0315486fe3ed4241bff355afa6d87e16 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0315486fe3ed4241bff355afa6d87e16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0315486fe3ed4241bff355afa6d87e16" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 38357 } } }
I20260926 16:36:16.067615 31091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0315486fe3ed4241bff355afa6d87e16 [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:16.067615 31092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0315486fe3ed4241bff355afa6d87e16 [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:16.068074 31099 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:36:16.070772 31099 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:36:16.076188 31099 catalog_manager.cc:1347] Generated new cluster ID: 9566b631ee2d4368a6872af66b5916e6
I20260926 16:36:16.076246 31099 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:36:16.099364 31099 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:36:16.100617 31099 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:36:16.107975 31099 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0315486fe3ed4241bff355afa6d87e16: Generated new TSK 0
I20260926 16:36:16.108757 31099 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:36:16.116256 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.193:0
--local_ip_for_outbound_sockets=127.11.8.193
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:38357
--builtin_ntp_servers=127.11.8.212:42743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 16:36:16.228072 31109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:16.228324 31109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:16.228396 31109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:16.232422 31109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:16.232491 31109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:16.232574 31109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.193
I20260926 16:36:16.237075 31109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:42743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.193
--webserver_port=0
--tserver_master_addrs=127.11.8.254:38357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.193
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:16.238090 31109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:16.239447 31109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:16.248385 31117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:16.248363 31115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:16.248550 31109 server_base.cc:1034] running on GCE node
W20260926 16:36:16.248710 31114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:16.249034 31109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:16.249634 31109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:16.250802 31109 hybrid_clock.cc:648] HybridClock initialized: now 1790440576250768 us; error 47 us; skew 500 ppm
I20260926 16:36:16.252504 31109 webserver.cc:460] Webserver started at http://127.11.8.193:35785/ using document root <none> and password file <none>
I20260926 16:36:16.253037 31109 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:16.253123 31109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:16.253335 31109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:16.255262 31109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-0/data/instance:
uuid: "e088a12b2f374b35afb0e07b13ce1113"
format_stamp: "Formatted at 2026-09-26 16:36:16 on dist-test-slave-2zvq"
server_key: "744fdd990868ddb2a0dd48bba348057f"
server_key_iv: "8dbfd95e3ad8c3df8d0b3dc0fd340a30"
server_key_version: "encryptionkey@0"
I20260926 16:36:16.255796 31109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance:
uuid: "e088a12b2f374b35afb0e07b13ce1113"
format_stamp: "Formatted at 2026-09-26 16:36:16 on dist-test-slave-2zvq"
server_key: "744fdd990868ddb2a0dd48bba348057f"
server_key_iv: "8dbfd95e3ad8c3df8d0b3dc0fd340a30"
server_key_version: "encryptionkey@0"
I20260926 16:36:16.259104 31109 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 16:36:16.261255 31123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:16.262357 31109 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:36:16.262491 31109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-0/wal
uuid: "e088a12b2f374b35afb0e07b13ce1113"
format_stamp: "Formatted at 2026-09-26 16:36:16 on dist-test-slave-2zvq"
server_key: "744fdd990868ddb2a0dd48bba348057f"
server_key_iv: "8dbfd95e3ad8c3df8d0b3dc0fd340a30"
server_key_version: "encryptionkey@0"
I20260926 16:36:16.262593 31109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:16.273209 31109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:16.273805 31109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:16.273988 31109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:16.274605 31109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:36:16.275467 31109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:36:16.275535 31109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:16.275605 31109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:36:16.275647 31109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:16.284101 31109 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.193:46791
I20260926 16:36:16.284128 31236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.193:46791 every 8 connection(s)
I20260926 16:36:16.284940 31109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-0/data/info.pb
I20260926 16:36:16.292140 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 31109
I20260926 16:36:16.292290 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-0/wal/instance
I20260926 16:36:16.292636 11299 external_mini_cluster.cc:1428] Setting key 5e65f7b32242f7988af7629189622f55
I20260926 16:36:16.294062 31088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:16.286774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:40263 (local address 127.11.8.254:38357)
0926 16:36:16.286916 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:16.286920 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:16.287967 (+ 1047us) server_negotiation.cc:407] Connection header received
0926 16:36:16.289283 (+ 1316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:16.289287 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:16.289347 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:16.289434 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:16.290311 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:16.290844 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:16.291986 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:16.292209 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:16.292321 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:16.293546 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:16.293548 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:16.293550 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:36:16.293562 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:16.293580 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:36:16.293587 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:16.293655 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:16.293868 (+ 213us) server_negotiation.cc:299] Negotiation successful
0926 16:36:16.293941 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 16:36:16.294749 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.194:0
--local_ip_for_outbound_sockets=127.11.8.194
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:38357
--builtin_ntp_servers=127.11.8.212:42743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 16:36:16.294991 31239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:16.287211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:38357 (local address 127.11.8.193:40263)
0926 16:36:16.287821 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:16.287862 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:16.289073 (+ 1211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:16.289597 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:16.289604 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:16.289683 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:36:16.290145 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:16.290157 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:16.290985 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:16.290989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:16.291848 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:16.291858 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:16.292378 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:16.292385 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:16.292593 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:16.293253 (+ 660us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:36:16.293255 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:36:16.293292 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:36:16.293373 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:36:16.293678 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:36:16.293686 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:36:16.293691 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 16:36:16.293814 (+ 123us) client_negotiation.cc:240] Negotiation successful
0926 16:36:16.294136 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":106,"threads_started":1}
I20260926 16:36:16.296116 31237 heartbeater.cc:347] Connected to a master server at 127.11.8.254:38357
I20260926 16:36:16.296365 31237 heartbeater.cc:464] Registering TS with master...
I20260926 16:36:16.296944 31237 heartbeater.cc:511] Master 127.11.8.254:38357 requested a full tablet report, sending...
I20260926 16:36:16.298554 31049 ts_manager.cc:194] Registered new tserver with Master: e088a12b2f374b35afb0e07b13ce1113 (127.11.8.193:46791)
I20260926 16:36:16.300298 31049 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.8.193:40263
W20260926 16:36:16.402091 31240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:16.402350 31240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:16.402388 31240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:16.406179 31240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:16.406222 31240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:16.406376 31240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.194
I20260926 16:36:16.410920 31240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:42743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.194
--webserver_port=0
--tserver_master_addrs=127.11.8.254:38357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.194
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:16.411895 31240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:16.413193 31240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:16.422593 31248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:16.422679 31246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:16.422610 31245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:16.422842 31240 server_base.cc:1034] running on GCE node
I20260926 16:36:16.423188 31240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:16.423771 31240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:16.424954 31240 hybrid_clock.cc:648] HybridClock initialized: now 1790440576424942 us; error 30 us; skew 500 ppm
I20260926 16:36:16.431591 31240 webserver.cc:460] Webserver started at http://127.11.8.194:38533/ using document root <none> and password file <none>
I20260926 16:36:16.432127 31240 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:16.432214 31240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:16.432454 31240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:16.434443 31240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-1/data/instance:
uuid: "f2dbdf416ec342d489aa940cd928eb6e"
format_stamp: "Formatted at 2026-09-26 16:36:16 on dist-test-slave-2zvq"
server_key: "b5f21ccbcdfd568dd5ce891333f66ace"
server_key_iv: "6d869d202fbdc99fc1da2dbc0646abc3"
server_key_version: "encryptionkey@0"
I20260926 16:36:16.434983 31240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance:
uuid: "f2dbdf416ec342d489aa940cd928eb6e"
format_stamp: "Formatted at 2026-09-26 16:36:16 on dist-test-slave-2zvq"
server_key: "b5f21ccbcdfd568dd5ce891333f66ace"
server_key_iv: "6d869d202fbdc99fc1da2dbc0646abc3"
server_key_version: "encryptionkey@0"
I20260926 16:36:16.438591 31240 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:36:16.440938 31254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:16.442109 31240 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:36:16.442245 31240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-1/wal
uuid: "f2dbdf416ec342d489aa940cd928eb6e"
format_stamp: "Formatted at 2026-09-26 16:36:16 on dist-test-slave-2zvq"
server_key: "b5f21ccbcdfd568dd5ce891333f66ace"
server_key_iv: "6d869d202fbdc99fc1da2dbc0646abc3"
server_key_version: "encryptionkey@0"
I20260926 16:36:16.442431 31240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:16.459702 31240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:16.460294 31240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:16.460480 31240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:16.461022 31240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:36:16.461903 31240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:36:16.461975 31240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:16.462044 31240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:36:16.462082 31240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:16.471282 31240 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.194:37117
I20260926 16:36:16.471311 31367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.194:37117 every 8 connection(s)
I20260926 16:36:16.472152 31240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-1/data/info.pb
I20260926 16:36:16.480777 31088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:16.473876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.194:53057 (local address 127.11.8.254:38357)
0926 16:36:16.474012 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:16.474015 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:16.475099 (+ 1084us) server_negotiation.cc:407] Connection header received
0926 16:36:16.476405 (+ 1306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:16.476408 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:16.476454 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:16.476530 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:16.477339 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:16.477866 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:16.479008 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:16.479242 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:16.479368 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:16.480356 (+ 988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:16.480359 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:16.480361 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:36:16.480373 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:16.480391 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:36:16.480398 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:16.480477 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:16.480617 (+ 140us) server_negotiation.cc:299] Negotiation successful
0926 16:36:16.480668 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 16:36:16.481237 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 31240
I20260926 16:36:16.481380 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-1/wal/instance
I20260926 16:36:16.481649 31370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:16.474249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:38357 (local address 127.11.8.194:53057)
0926 16:36:16.474953 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:16.474993 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:16.476223 (+ 1230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:16.476682 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:16.476689 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:16.476769 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:36:16.477219 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:16.477231 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:16.477968 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:16.477971 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:16.478889 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:16.478898 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:16.479435 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:16.479437 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:16.479569 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:16.480100 (+ 531us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:36:16.480101 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:36:16.480139 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:36:16.480197 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:36:16.480490 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:36:16.480497 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:36:16.480501 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 16:36:16.480591 (+ 90us) client_negotiation.cc:240] Negotiation successful
0926 16:36:16.480802 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":109,"threads_started":1}
I20260926 16:36:16.481681 11299 external_mini_cluster.cc:1428] Setting key 9fd836e1e7d77ca7ffe4a33919dc40e4
I20260926 16:36:16.482762 31368 heartbeater.cc:347] Connected to a master server at 127.11.8.254:38357
I20260926 16:36:16.482988 31368 heartbeater.cc:464] Registering TS with master...
I20260926 16:36:16.483480 31368 heartbeater.cc:511] Master 127.11.8.254:38357 requested a full tablet report, sending...
I20260926 16:36:16.483496 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.195:0
--local_ip_for_outbound_sockets=127.11.8.195
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:38357
--builtin_ntp_servers=127.11.8.212:42743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 16:36:16.484462 31049 ts_manager.cc:194] Registered new tserver with Master: f2dbdf416ec342d489aa940cd928eb6e (127.11.8.194:37117)
I20260926 16:36:16.485000 31049 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.8.194:53057
W20260926 16:36:16.591691 31371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:16.591964 31371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:16.592033 31371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:16.596501 31371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:16.596611 31371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:16.596727 31371 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.195
I20260926 16:36:16.602288 31371 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:42743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.195
--webserver_port=0
--tserver_master_addrs=127.11.8.254:38357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.195
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:16.603456 31371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:16.604915 31371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 16:36:16.614897 31371 server_base.cc:1034] running on GCE node
W20260926 16:36:16.614905 31376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:16.614905 31377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:16.615090 31379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:16.615608 31371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:16.616204 31371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:16.617414 31371 hybrid_clock.cc:648] HybridClock initialized: now 1790440576617390 us; error 35 us; skew 500 ppm
I20260926 16:36:16.619262 31371 webserver.cc:460] Webserver started at http://127.11.8.195:42231/ using document root <none> and password file <none>
I20260926 16:36:16.619885 31371 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:16.619975 31371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:16.620200 31371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:16.622256 31371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-2/data/instance:
uuid: "79a1774d5a94481397fbe189750607c8"
format_stamp: "Formatted at 2026-09-26 16:36:16 on dist-test-slave-2zvq"
server_key: "b585a7e87fcb4dec83c2ea6b6bae83b0"
server_key_iv: "76dfaaf4c05ba83cfd850b9afb224e0c"
server_key_version: "encryptionkey@0"
I20260926 16:36:16.622908 31371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance:
uuid: "79a1774d5a94481397fbe189750607c8"
format_stamp: "Formatted at 2026-09-26 16:36:16 on dist-test-slave-2zvq"
server_key: "b585a7e87fcb4dec83c2ea6b6bae83b0"
server_key_iv: "76dfaaf4c05ba83cfd850b9afb224e0c"
server_key_version: "encryptionkey@0"
I20260926 16:36:16.626446 31371 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260926 16:36:16.628934 31385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:16.630131 31371 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:36:16.630297 31371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-2/wal
uuid: "79a1774d5a94481397fbe189750607c8"
format_stamp: "Formatted at 2026-09-26 16:36:16 on dist-test-slave-2zvq"
server_key: "b585a7e87fcb4dec83c2ea6b6bae83b0"
server_key_iv: "76dfaaf4c05ba83cfd850b9afb224e0c"
server_key_version: "encryptionkey@0"
I20260926 16:36:16.630466 31371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:16.662907 31371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:16.663657 31371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:16.663853 31371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:16.664443 31371 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 16:36:16.665419 31371 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 16:36:16.665513 31371 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:16.665623 31371 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 16:36:16.665663 31371 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:16.675112 31371 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.195:33777
I20260926 16:36:16.675153 31498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.195:33777 every 8 connection(s)
I20260926 16:36:16.676092 31371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-2/data/info.pb
I20260926 16:36:16.680761 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 31371
I20260926 16:36:16.680884 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-2/wal/instance
I20260926 16:36:16.681258 11299 external_mini_cluster.cc:1428] Setting key 9faf8dc255e167c6a9e8c0414184a99a
I20260926 16:36:16.686619 31088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:16.677954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.195:47899 (local address 127.11.8.254:38357)
0926 16:36:16.678099 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:16.678104 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:16.679334 (+ 1230us) server_negotiation.cc:407] Connection header received
0926 16:36:16.681012 (+ 1678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:16.681015 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:16.681062 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:16.681128 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:16.682415 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:16.683243 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:16.684622 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:16.684869 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:16.684962 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:16.686063 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:16.686067 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:16.686069 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:36:16.686080 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:16.686099 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:36:16.686106 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:16.686175 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:16.686457 (+ 282us) server_negotiation.cc:299] Negotiation successful
0926 16:36:16.686511 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 16:36:16.687507 31501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:16.678468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:38357 (local address 127.11.8.195:47899)
0926 16:36:16.679157 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:16.679218 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:16.680771 (+ 1553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:16.681340 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:16.681351 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:16.681465 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:36:16.682199 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:16.682220 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:16.683407 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:16.683413 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:16.684497 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:16.684507 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:16.685025 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:16.685028 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:16.685178 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:16.685797 (+ 619us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 16:36:16.685799 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:36:16.685841 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 16:36:16.685907 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:36:16.686220 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:36:16.686232 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:36:16.686239 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 16:36:16.686435 (+ 196us) client_negotiation.cc:240] Negotiation successful
0926 16:36:16.686668 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":112,"threads_started":1}
I20260926 16:36:16.688632 31499 heartbeater.cc:347] Connected to a master server at 127.11.8.254:38357
I20260926 16:36:16.688902 31499 heartbeater.cc:464] Registering TS with master...
I20260926 16:36:16.689431 31499 heartbeater.cc:511] Master 127.11.8.254:38357 requested a full tablet report, sending...
I20260926 16:36:16.690547 31049 ts_manager.cc:194] Registered new tserver with Master: 79a1774d5a94481397fbe189750607c8 (127.11.8.195:33777)
I20260926 16:36:16.691140 31049 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.8.195:47899
I20260926 16:36:16.695331 11299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 16:36:16.701152 31088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:16.697303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60302 (local address 127.11.8.254:38357)
0926 16:36:16.697451 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:16.697456 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:16.697597 (+ 141us) server_negotiation.cc:407] Connection header received
0926 16:36:16.697760 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:16.697763 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:16.697807 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:16.697899 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:16.698169 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:16.698761 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:16.699616 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:16.699857 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:16.699965 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:16.700698 (+ 733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:16.700701 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:16.700704 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:36:16.700716 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:16.700751 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:36:16.700759 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:16.700835 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:16.700955 (+ 120us) server_negotiation.cc:299] Negotiation successful
0926 16:36:16.701006 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 16:36:16.703874 31049 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60302:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 16:36:16.706840 31049 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 16:36:16.721007 31514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:16.715498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37808 (local address 127.11.8.194:37117)
0926 16:36:16.715868 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:16.715883 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:16.716229 (+ 346us) server_negotiation.cc:407] Connection header received
0926 16:36:16.716334 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:16.716340 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:16.716544 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:16.717034 (+ 490us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:36:16.717171 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:16.718259 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:16.719844 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:16.720350 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:16.720469 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:16.720654 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:16.720744 (+ 90us) server_negotiation.cc:299] Negotiation successful
0926 16:36:16.720853 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":152,"threads_started":1}
I20260926 16:36:16.721002 31513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:16.714972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:37117 (local address 127.0.0.1:37808)
0926 16:36:16.716054 (+ 1082us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:16.716068 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:16.716155 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:16.716706 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:16.716710 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:16.716748 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:36:16.716903 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:16.716910 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:16.718409 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:16.718412 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:16.719717 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:16.719729 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:16.720470 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:16.720473 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:16.720580 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:16.720616 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 16:36:16.720718 (+ 102us) client_negotiation.cc:240] Negotiation successful
0926 16:36:16.720848 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1005,"mutex_wait_us":73,"spinlock_wait_cycles":5888,"thread_start_us":94,"threads_started":1}
I20260926 16:36:16.720973 31510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:16.714819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:33777 (local address 127.0.0.1:54724)
0926 16:36:16.715599 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:16.715652 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:16.715802 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:16.716758 (+ 956us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:16.716761 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:16.716773 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:36:16.716983 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:16.716991 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:16.718374 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:16.718377 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:16.719447 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:16.719457 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:16.720518 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:16.720522 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:16.720605 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:16.720628 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 16:36:16.720691 (+ 63us) client_negotiation.cc:240] Negotiation successful
0926 16:36:16.720840 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"mutex_wait_us":201,"thread_start_us":80,"threads_started":1}
I20260926 16:36:16.721781 31512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:16.714976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54724 (local address 127.11.8.195:33777)
0926 16:36:16.716388 (+ 1412us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:16.716399 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:16.716418 (+ 19us) server_negotiation.cc:407] Connection header received
0926 16:36:16.716479 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:16.716483 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:16.716628 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:16.716901 (+ 273us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:36:16.717079 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:16.718252 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:16.719573 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:16.720380 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:16.721103 (+ 723us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:16.721448 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:16.721502 (+ 54us) server_negotiation.cc:299] Negotiation successful
0926 16:36:16.721592 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1321,"thread_start_us":124,"threads_started":1}
I20260926 16:36:16.722543 31511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:16.715197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:46791 (local address 127.0.0.1:58310)
0926 16:36:16.716168 (+ 971us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:16.716182 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:16.716272 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:16.716682 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:16.716688 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:16.716709 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:36:16.716983 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:16.716991 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:16.719647 (+ 2656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:16.719650 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:16.720727 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:16.720734 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:16.722168 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:16.722171 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:16.722234 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:16.722250 (+ 16us) client_negotiation.cc:769] Sending connection context
0926 16:36:16.722359 (+ 109us) client_negotiation.cc:240] Negotiation successful
0926 16:36:16.722424 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":906,"mutex_wait_us":77,"thread_start_us":103,"threads_started":1}
I20260926 16:36:16.722651 31515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:16.715916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58310 (local address 127.11.8.193:46791)
0926 16:36:16.716238 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:16.716251 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:16.716285 (+ 34us) server_negotiation.cc:407] Connection header received
0926 16:36:16.716370 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:16.716382 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:16.716526 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:16.716680 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:36:16.718412 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:16.719558 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:16.721344 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:16.722066 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:16.722180 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:16.722380 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:16.722426 (+ 46us) server_negotiation.cc:299] Negotiation successful
0926 16:36:16.722508 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":116,"threads_started":1}
I20260926 16:36:16.724481 31433 tablet_service.cc:1467] Processing CreateTablet for tablet cd808553c4584e51bea2a1f1a18f9633 (DEFAULT_TABLE table=test-table [id=eb659d984af54cb2a8d71577fcdb5d64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:36:16.724522 31302 tablet_service.cc:1467] Processing CreateTablet for tablet cd808553c4584e51bea2a1f1a18f9633 (DEFAULT_TABLE table=test-table [id=eb659d984af54cb2a8d71577fcdb5d64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:36:16.725104 31171 tablet_service.cc:1467] Processing CreateTablet for tablet cd808553c4584e51bea2a1f1a18f9633 (DEFAULT_TABLE table=test-table [id=eb659d984af54cb2a8d71577fcdb5d64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 16:36:16.725567 31433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd808553c4584e51bea2a1f1a18f9633. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:16.725616 31302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd808553c4584e51bea2a1f1a18f9633. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:16.725970 31171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd808553c4584e51bea2a1f1a18f9633. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:16.731721 31516 tablet_bootstrap.cc:492] T cd808553c4584e51bea2a1f1a18f9633 P 79a1774d5a94481397fbe189750607c8: Bootstrap starting.
I20260926 16:36:16.733147 31518 tablet_bootstrap.cc:492] T cd808553c4584e51bea2a1f1a18f9633 P f2dbdf416ec342d489aa940cd928eb6e: Bootstrap starting.
I20260926 16:36:16.733172 31517 tablet_bootstrap.cc:492] T cd808553c4584e51bea2a1f1a18f9633 P e088a12b2f374b35afb0e07b13ce1113: Bootstrap starting.
I20260926 16:36:16.734225 31516 tablet_bootstrap.cc:654] T cd808553c4584e51bea2a1f1a18f9633 P 79a1774d5a94481397fbe189750607c8: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:16.734980 31517 tablet_bootstrap.cc:654] T cd808553c4584e51bea2a1f1a18f9633 P e088a12b2f374b35afb0e07b13ce1113: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:16.735091 31516 log.cc:824] T cd808553c4584e51bea2a1f1a18f9633 P 79a1774d5a94481397fbe189750607c8: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:16.735282 31518 tablet_bootstrap.cc:654] T cd808553c4584e51bea2a1f1a18f9633 P f2dbdf416ec342d489aa940cd928eb6e: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:16.735865 31517 log.cc:824] T cd808553c4584e51bea2a1f1a18f9633 P e088a12b2f374b35afb0e07b13ce1113: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:16.736056 31518 log.cc:824] T cd808553c4584e51bea2a1f1a18f9633 P f2dbdf416ec342d489aa940cd928eb6e: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:16.737556 31516 tablet_bootstrap.cc:492] T cd808553c4584e51bea2a1f1a18f9633 P 79a1774d5a94481397fbe189750607c8: No bootstrap required, opened a new log
I20260926 16:36:16.737563 31517 tablet_bootstrap.cc:492] T cd808553c4584e51bea2a1f1a18f9633 P e088a12b2f374b35afb0e07b13ce1113: No bootstrap required, opened a new log
I20260926 16:36:16.737767 31517 ts_tablet_manager.cc:1397] T cd808553c4584e51bea2a1f1a18f9633 P e088a12b2f374b35afb0e07b13ce1113: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260926 16:36:16.737763 31516 ts_tablet_manager.cc:1397] T cd808553c4584e51bea2a1f1a18f9633 P 79a1774d5a94481397fbe189750607c8: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260926 16:36:16.739761 31518 tablet_bootstrap.cc:492] T cd808553c4584e51bea2a1f1a18f9633 P f2dbdf416ec342d489aa940cd928eb6e: No bootstrap required, opened a new log
I20260926 16:36:16.739940 31518 ts_tablet_manager.cc:1397] T cd808553c4584e51bea2a1f1a18f9633 P f2dbdf416ec342d489aa940cd928eb6e: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260926 16:36:16.741253 31517 raft_consensus.cc:348] T cd808553c4584e51bea2a1f1a18f9633 P e088a12b2f374b35afb0e07b13ce1113 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79a1774d5a94481397fbe189750607c8" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33777 } } peers { permanent_uuid: "f2dbdf416ec342d489aa940cd928eb6e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 37117 } } peers { permanent_uuid: "e088a12b2f374b35afb0e07b13ce1113" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46791 } }
I20260926 16:36:16.741495 31517 raft_consensus.cc:374] T cd808553c4584e51bea2a1f1a18f9633 P e088a12b2f374b35afb0e07b13ce1113 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:36:16.741566 31517 raft_consensus.cc:729] T cd808553c4584e51bea2a1f1a18f9633 P e088a12b2f374b35afb0e07b13ce1113 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e088a12b2f374b35afb0e07b13ce1113, State: Initialized, Role: FOLLOWER
I20260926 16:36:16.742017 31517 consensus_queue.cc:260] T cd808553c4584e51bea2a1f1a18f9633 P e088a12b2f374b35afb0e07b13ce1113 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79a1774d5a94481397fbe189750607c8" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33777 } } peers { permanent_uuid: "f2dbdf416ec342d489aa940cd928eb6e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 37117 } } peers { permanent_uuid: "e088a12b2f374b35afb0e07b13ce1113" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46791 } }
I20260926 16:36:16.741976 31516 raft_consensus.cc:348] T cd808553c4584e51bea2a1f1a18f9633 P 79a1774d5a94481397fbe189750607c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79a1774d5a94481397fbe189750607c8" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33777 } } peers { permanent_uuid: "f2dbdf416ec342d489aa940cd928eb6e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 37117 } } peers { permanent_uuid: "e088a12b2f374b35afb0e07b13ce1113" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46791 } }
I20260926 16:36:16.742242 31516 raft_consensus.cc:374] T cd808553c4584e51bea2a1f1a18f9633 P 79a1774d5a94481397fbe189750607c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:36:16.742364 31516 raft_consensus.cc:729] T cd808553c4584e51bea2a1f1a18f9633 P 79a1774d5a94481397fbe189750607c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79a1774d5a94481397fbe189750607c8, State: Initialized, Role: FOLLOWER
I20260926 16:36:16.742631 31237 heartbeater.cc:503] Master 127.11.8.254:38357 was elected leader, sending a full tablet report...
I20260926 16:36:16.742861 31516 consensus_queue.cc:260] T cd808553c4584e51bea2a1f1a18f9633 P 79a1774d5a94481397fbe189750607c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79a1774d5a94481397fbe189750607c8" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33777 } } peers { permanent_uuid: "f2dbdf416ec342d489aa940cd928eb6e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 37117 } } peers { permanent_uuid: "e088a12b2f374b35afb0e07b13ce1113" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46791 } }
I20260926 16:36:16.743005 31517 ts_tablet_manager.cc:1428] T cd808553c4584e51bea2a1f1a18f9633 P e088a12b2f374b35afb0e07b13ce1113: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260926 16:36:16.743628 31499 heartbeater.cc:503] Master 127.11.8.254:38357 was elected leader, sending a full tablet report...
I20260926 16:36:16.743752 31516 ts_tablet_manager.cc:1428] T cd808553c4584e51bea2a1f1a18f9633 P 79a1774d5a94481397fbe189750607c8: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260926 16:36:16.744730 31518 raft_consensus.cc:348] T cd808553c4584e51bea2a1f1a18f9633 P f2dbdf416ec342d489aa940cd928eb6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79a1774d5a94481397fbe189750607c8" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33777 } } peers { permanent_uuid: "f2dbdf416ec342d489aa940cd928eb6e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 37117 } } peers { permanent_uuid: "e088a12b2f374b35afb0e07b13ce1113" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46791 } }
I20260926 16:36:16.744976 31518 raft_consensus.cc:374] T cd808553c4584e51bea2a1f1a18f9633 P f2dbdf416ec342d489aa940cd928eb6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:36:16.745131 31518 raft_consensus.cc:729] T cd808553c4584e51bea2a1f1a18f9633 P f2dbdf416ec342d489aa940cd928eb6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2dbdf416ec342d489aa940cd928eb6e, State: Initialized, Role: FOLLOWER
I20260926 16:36:16.745582 31518 consensus_queue.cc:260] T cd808553c4584e51bea2a1f1a18f9633 P f2dbdf416ec342d489aa940cd928eb6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79a1774d5a94481397fbe189750607c8" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33777 } } peers { permanent_uuid: "f2dbdf416ec342d489aa940cd928eb6e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 37117 } } peers { permanent_uuid: "e088a12b2f374b35afb0e07b13ce1113" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46791 } }
I20260926 16:36:16.746434 31368 heartbeater.cc:503] Master 127.11.8.254:38357 was elected leader, sending a full tablet report...
I20260926 16:36:16.746594 31518 ts_tablet_manager.cc:1428] T cd808553c4584e51bea2a1f1a18f9633 P f2dbdf416ec342d489aa940cd928eb6e: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
W20260926 16:36:16.786058 31238 tablet.cc:2369] T cd808553c4584e51bea2a1f1a18f9633 P e088a12b2f374b35afb0e07b13ce1113: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:36:16.829519 31522 raft_consensus.cc:482] T cd808553c4584e51bea2a1f1a18f9633 P e088a12b2f374b35afb0e07b13ce1113 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 16:36:16.829800 31522 raft_consensus.cc:504] T cd808553c4584e51bea2a1f1a18f9633 P e088a12b2f374b35afb0e07b13ce1113 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79a1774d5a94481397fbe189750607c8" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33777 } } peers { permanent_uuid: "f2dbdf416ec342d489aa940cd928eb6e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 37117 } } peers { permanent_uuid: "e088a12b2f374b35afb0e07b13ce1113" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46791 } }
I20260926 16:36:16.831058 31522 leader_election.cc:288] T cd808553c4584e51bea2a1f1a18f9633 P e088a12b2f374b35afb0e07b13ce1113 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79a1774d5a94481397fbe189750607c8 (127.11.8.195:33777), f2dbdf416ec342d489aa940cd928eb6e (127.11.8.194:37117)
I20260926 16:36:16.836655 31514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:16.831437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:38085 (local address 127.11.8.194:37117)
0926 16:36:16.831586 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:16.831591 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:16.831780 (+ 189us) server_negotiation.cc:407] Connection header received
0926 16:36:16.831942 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:16.831947 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:16.832005 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:16.832080 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:36:16.832485 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:16.833381 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:16.835239 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:16.835906 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:16.836046 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:16.836099 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:16.836443 (+ 344us) server_negotiation.cc:299] Negotiation successful
0926 16:36:16.836518 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 16:36:16.836655 31512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:16.831569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.193:56991 (local address 127.11.8.195:33777)
0926 16:36:16.831725 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:16.831730 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:16.831875 (+ 145us) server_negotiation.cc:407] Connection header received
0926 16:36:16.832048 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:16.832054 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:16.832110 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:16.832186 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 16:36:16.832586 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:16.833435 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:16.835239 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:16.835906 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:16.836046 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:16.836099 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:16.836443 (+ 344us) server_negotiation.cc:299] Negotiation successful
0926 16:36:16.836518 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 16:36:16.836709 31525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:16.831320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:37117 (local address 127.11.8.193:38085)
0926 16:36:16.831682 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:16.831694 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:16.831823 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:16.832110 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:16.832114 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:16.832133 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:36:16.832353 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:16.832361 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:16.833536 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:16.833539 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:16.835105 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:16.835112 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:16.836072 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:16.836076 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:16.836228 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:16.836322 (+ 94us) client_negotiation.cc:769] Sending connection context
0926 16:36:16.836409 (+ 87us) client_negotiation.cc:240] Negotiation successful
0926 16:36:16.836530 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"mutex_wait_us":210,"thread_start_us":80,"threads_started":1}
I20260926 16:36:16.836709 31526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:16.831320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.195:33777 (local address 127.11.8.193:56991)
0926 16:36:16.831767 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:16.831789 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:16.831923 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:16.832222 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:16.832228 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:16.832250 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 16:36:16.832407 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:16.832416 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:16.833535 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:16.833539 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:16.835102 (+ 1563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:16.835113 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:16.836080 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:16.836083 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:16.836230 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:16.836326 (+ 96us) client_negotiation.cc:769] Sending connection context
0926 16:36:16.836409 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 16:36:16.836520 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"mutex_wait_us":306,"thread_start_us":91,"threads_started":1}
I20260926 16:36:16.837700 31322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd808553c4584e51bea2a1f1a18f9633" candidate_uuid: "e088a12b2f374b35afb0e07b13ce1113" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2dbdf416ec342d489aa940cd928eb6e" is_pre_election: true
I20260926 16:36:16.837751 31453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd808553c4584e51bea2a1f1a18f9633" candidate_uuid: "e088a12b2f374b35afb0e07b13ce1113" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79a1774d5a94481397fbe189750607c8" is_pre_election: true
I20260926 16:36:16.838025 31322 raft_consensus.cc:2454] T cd808553c4584e51bea2a1f1a18f9633 P f2dbdf416ec342d489aa940cd928eb6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e088a12b2f374b35afb0e07b13ce1113 in term 0.
I20260926 16:36:16.838025 31453 raft_consensus.cc:2454] T cd808553c4584e51bea2a1f1a18f9633 P 79a1774d5a94481397fbe189750607c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e088a12b2f374b35afb0e07b13ce1113 in term 0.
I20260926 16:36:16.838519 31127 leader_election.cc:302] T cd808553c4584e51bea2a1f1a18f9633 P e088a12b2f374b35afb0e07b13ce1113 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e088a12b2f374b35afb0e07b13ce1113, f2dbdf416ec342d489aa940cd928eb6e; no voters:
I20260926 16:36:16.838780 31522 raft_consensus.cc:2781] T cd808553c4584e51bea2a1f1a18f9633 P e088a12b2f374b35afb0e07b13ce1113 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 16:36:16.838881 31522 raft_consensus.cc:482] T cd808553c4584e51bea2a1f1a18f9633 P e088a12b2f374b35afb0e07b13ce1113 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 16:36:16.838941 31522 raft_consensus.cc:3037] T cd808553c4584e51bea2a1f1a18f9633 P e088a12b2f374b35afb0e07b13ce1113 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:16.839965 31522 raft_consensus.cc:504] T cd808553c4584e51bea2a1f1a18f9633 P e088a12b2f374b35afb0e07b13ce1113 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79a1774d5a94481397fbe189750607c8" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33777 } } peers { permanent_uuid: "f2dbdf416ec342d489aa940cd928eb6e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 37117 } } peers { permanent_uuid: "e088a12b2f374b35afb0e07b13ce1113" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46791 } }
I20260926 16:36:16.840299 31522 leader_election.cc:288] T cd808553c4584e51bea2a1f1a18f9633 P e088a12b2f374b35afb0e07b13ce1113 [CANDIDATE]: Term 1 election: Requested vote from peers 79a1774d5a94481397fbe189750607c8 (127.11.8.195:33777), f2dbdf416ec342d489aa940cd928eb6e (127.11.8.194:37117)
I20260926 16:36:16.840626 31322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd808553c4584e51bea2a1f1a18f9633" candidate_uuid: "e088a12b2f374b35afb0e07b13ce1113" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2dbdf416ec342d489aa940cd928eb6e"
I20260926 16:36:16.840628 31453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd808553c4584e51bea2a1f1a18f9633" candidate_uuid: "e088a12b2f374b35afb0e07b13ce1113" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79a1774d5a94481397fbe189750607c8"
I20260926 16:36:16.840767 31322 raft_consensus.cc:3037] T cd808553c4584e51bea2a1f1a18f9633 P f2dbdf416ec342d489aa940cd928eb6e [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:16.840764 31453 raft_consensus.cc:3037] T cd808553c4584e51bea2a1f1a18f9633 P 79a1774d5a94481397fbe189750607c8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:16.841697 31322 raft_consensus.cc:2454] T cd808553c4584e51bea2a1f1a18f9633 P f2dbdf416ec342d489aa940cd928eb6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e088a12b2f374b35afb0e07b13ce1113 in term 1.
I20260926 16:36:16.841825 31453 raft_consensus.cc:2454] T cd808553c4584e51bea2a1f1a18f9633 P 79a1774d5a94481397fbe189750607c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e088a12b2f374b35afb0e07b13ce1113 in term 1.
I20260926 16:36:16.842000 31127 leader_election.cc:302] T cd808553c4584e51bea2a1f1a18f9633 P e088a12b2f374b35afb0e07b13ce1113 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e088a12b2f374b35afb0e07b13ce1113, f2dbdf416ec342d489aa940cd928eb6e; no voters:
I20260926 16:36:16.842173 31522 raft_consensus.cc:2781] T cd808553c4584e51bea2a1f1a18f9633 P e088a12b2f374b35afb0e07b13ce1113 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:36:16.842407 31522 raft_consensus.cc:686] T cd808553c4584e51bea2a1f1a18f9633 P e088a12b2f374b35afb0e07b13ce1113 [term 1 LEADER]: Becoming Leader. State: Replica: e088a12b2f374b35afb0e07b13ce1113, State: Running, Role: LEADER
I20260926 16:36:16.842749 31522 consensus_queue.cc:237] T cd808553c4584e51bea2a1f1a18f9633 P e088a12b2f374b35afb0e07b13ce1113 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79a1774d5a94481397fbe189750607c8" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33777 } } peers { permanent_uuid: "f2dbdf416ec342d489aa940cd928eb6e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 37117 } } peers { permanent_uuid: "e088a12b2f374b35afb0e07b13ce1113" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46791 } }
I20260926 16:36:16.845687 31049 catalog_manager.cc:5495] T cd808553c4584e51bea2a1f1a18f9633 P e088a12b2f374b35afb0e07b13ce1113 reported cstate change: term changed from 0 to 1, leader changed from <none> to e088a12b2f374b35afb0e07b13ce1113 (127.11.8.193). New cstate: current_term: 1 leader_uuid: "e088a12b2f374b35afb0e07b13ce1113" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79a1774d5a94481397fbe189750607c8" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2dbdf416ec342d489aa940cd928eb6e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 37117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e088a12b2f374b35afb0e07b13ce1113" member_type: VOTER last_known_addr { host: "127.11.8.193" port: 46791 } health_report { overall_health: HEALTHY } } }
I20260926 16:36:16.870452 31515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:16.866897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58314 (local address 127.11.8.193:46791)
0926 16:36:16.867062 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:16.867067 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:16.867081 (+ 14us) server_negotiation.cc:407] Connection header received
0926 16:36:16.867227 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:16.867230 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:16.867283 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:16.867389 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:36:16.867656 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:16.868199 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:16.869049 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:16.869260 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:16.869370 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:16.869666 (+ 296us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:36:16.870064 (+ 398us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:36:16.870196 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:16.870236 (+ 40us) server_negotiation.cc:299] Negotiation successful
0926 16:36:16.870309 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 16:36:16.876114 31322 raft_consensus.cc:1261] T cd808553c4584e51bea2a1f1a18f9633 P f2dbdf416ec342d489aa940cd928eb6e [term 1 FOLLOWER]: Refusing update from remote peer e088a12b2f374b35afb0e07b13ce1113: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:36:16.876113 31453 raft_consensus.cc:1261] T cd808553c4584e51bea2a1f1a18f9633 P 79a1774d5a94481397fbe189750607c8 [term 1 FOLLOWER]: Refusing update from remote peer e088a12b2f374b35afb0e07b13ce1113: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:36:16.876811 31527 consensus_queue.cc:1035] T cd808553c4584e51bea2a1f1a18f9633 P e088a12b2f374b35afb0e07b13ce1113 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2dbdf416ec342d489aa940cd928eb6e" member_type: VOTER last_known_addr { host: "127.11.8.194" port: 37117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:36:16.876991 31522 consensus_queue.cc:1035] T cd808553c4584e51bea2a1f1a18f9633 P e088a12b2f374b35afb0e07b13ce1113 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79a1774d5a94481397fbe189750607c8" member_type: VOTER last_known_addr { host: "127.11.8.195" port: 33777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:36:16.892686 31048 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60302:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:36:16.892886 31048 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60302:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 16:36:16.895103 31048 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0315486fe3ed4241bff355afa6d87e16: Sending DeleteTablet for 3 replicas of tablet cd808553c4584e51bea2a1f1a18f9633
I20260926 16:36:16.895820 31433 tablet_service.cc:1514] Processing DeleteTablet for tablet cd808553c4584e51bea2a1f1a18f9633 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:36:16 UTC) from {username='slave'} at 127.0.0.1:54724
I20260926 16:36:16.895864 31302 tablet_service.cc:1514] Processing DeleteTablet for tablet cd808553c4584e51bea2a1f1a18f9633 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:36:16 UTC) from {username='slave'} at 127.0.0.1:37808
I20260926 16:36:16.895864 31171 tablet_service.cc:1514] Processing DeleteTablet for tablet cd808553c4584e51bea2a1f1a18f9633 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 16:36:16 UTC) from {username='slave'} at 127.0.0.1:58310
I20260926 16:36:16.896368 31540 tablet_replica.cc:330] T cd808553c4584e51bea2a1f1a18f9633 P 79a1774d5a94481397fbe189750607c8: stopping tablet replica
I20260926 16:36:16.896519 31542 tablet_replica.cc:330] T cd808553c4584e51bea2a1f1a18f9633 P f2dbdf416ec342d489aa940cd928eb6e: stopping tablet replica
I20260926 16:36:16.896828 31542 raft_consensus.cc:2229] T cd808553c4584e51bea2a1f1a18f9633 P f2dbdf416ec342d489aa940cd928eb6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:36:16.897094 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 31109
I20260926 16:36:16.897217 31542 raft_consensus.cc:2258] T cd808553c4584e51bea2a1f1a18f9633 P f2dbdf416ec342d489aa940cd928eb6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:36:16.897846 31540 raft_consensus.cc:2229] T cd808553c4584e51bea2a1f1a18f9633 P 79a1774d5a94481397fbe189750607c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 16:36:16.898065 31540 raft_consensus.cc:2258] T cd808553c4584e51bea2a1f1a18f9633 P 79a1774d5a94481397fbe189750607c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 16:36:16.899312 31540 ts_tablet_manager.cc:1905] T cd808553c4584e51bea2a1f1a18f9633 P 79a1774d5a94481397fbe189750607c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:36:16.901412 31540 ts_tablet_manager.cc:1918] T cd808553c4584e51bea2a1f1a18f9633 P 79a1774d5a94481397fbe189750607c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 16:36:16.901481 31540 log.cc:1196] T cd808553c4584e51bea2a1f1a18f9633 P 79a1774d5a94481397fbe189750607c8: Deleting WAL directory at /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790440324823152-11299-0/minicluster-data/ts-2/wal/wals/cd808553c4584e51bea2a1f1a18f9633
I20260926 16:36:16.901692 31540 ts_tablet_manager.cc:1939] T cd808553c4584e51bea2a1f1a18f9633 P 79a1774d5a94481397fbe189750607c8: Deleting consensus metadata
I20260926 16:36:16.903478 31542 ts_tablet_manager.cc:1905] T cd808553c4584e51bea2a1f1a18f9633 P f2dbdf416ec342d489aa940cd928eb6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 16:36:16.903609 31035 catalog_manager.cc:4841] TS 79a1774d5a94481397fbe189750607c8 (127.11.8.195:33777): tablet cd808553c4584e51bea2a1f1a18f9633 (table test-table [id=eb659d984af54cb2a8d71577fcdb5d64]) successfully deleted
W20260926 16:36:16.903697 31034 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.8.193:46791 (error 108)
I20260926 16:36:16.904477 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 31240
I20260926 16:36:16.904703 31511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:16.904397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.193:46791 (local address 127.0.0.1:58330)
0926 16:36:16.904561 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:16.904631 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.193:46791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
W20260926 16:36:16.905944 31034 catalog_manager.cc:4584] TS e088a12b2f374b35afb0e07b13ce1113 (127.11.8.193:46791): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cd808553c4584e51bea2a1f1a18f9633: Network error: Client connection negotiation failed: client connection to 127.11.8.193:46791: connect: Connection refused (error 111)
I20260926 16:36:16.909884 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 31371
I20260926 16:36:16.911161 31511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:16.909987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.194:37117 (local address 127.0.0.1:37822)
0926 16:36:16.911003 (+ 1016us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:16.911067 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.194:37117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":918}
I20260926 16:36:16.915555 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 31018
2026-09-26T16:36:16Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1060 ms)
[----------] 6 tests from EncryptionPolicyTest (10658 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:36:16 dist-test-slave-2zvq krb5kdc[31546](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:36:16 dist-test-slave-2zvq krb5kdc[31546](info): set up 2 sockets
Sep 26 16:36:16 dist-test-slave-2zvq krb5kdc[31546](info): commencing operation
krb5kdc: starting...
W20260926 16:36:17.952083 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:36:18 dist-test-slave-2zvq krb5kdc[31546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:36:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:36:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 16:36:18.066489 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35813
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35813
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:36:18.175915 31559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:18.176213 31559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:18.176309 31559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:18.180303 31559 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:36:18.180388 31559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:18.180428 31559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:18.180487 31559 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:36:18.180514 31559 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:36:18.185598 31559 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35813
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:18.186599 31559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:18.187593 31559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 16:36:18.196141 31559 server_base.cc:1034] running on GCE node
W20260926 16:36:18.196086 31565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:18.196064 31567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:18.196089 31564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:18.196863 31559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:18.197902 31559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:18.199076 31559 hybrid_clock.cc:648] HybridClock initialized: now 1790440578199050 us; error 30 us; skew 500 ppm
Sep 26 16:36:18 dist-test-slave-2zvq krb5kdc[31546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440578, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:36:18.201916 31559 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 16:36:18.202792 31559 webserver.cc:460] Webserver started at http://127.0.0.1:41321/ using document root <none> and password file <none>
I20260926 16:36:18.203293 31559 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:18.203357 31559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:18.203517 31559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:18.205475 31559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "25d712683b9b4eee8c48a04daa6676f6"
format_stamp: "Formatted at 2026-09-26 16:36:18 on dist-test-slave-2zvq"
server_key: "f36c4a7e538361e38e486ad676829c3f"
server_key_iv: "9e03a3c693426c86163037a6e1fbcf4b"
server_key_version: "encryptionkey@0"
I20260926 16:36:18.205989 31559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "25d712683b9b4eee8c48a04daa6676f6"
format_stamp: "Formatted at 2026-09-26 16:36:18 on dist-test-slave-2zvq"
server_key: "f36c4a7e538361e38e486ad676829c3f"
server_key_iv: "9e03a3c693426c86163037a6e1fbcf4b"
server_key_version: "encryptionkey@0"
I20260926 16:36:18.209619 31559 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260926 16:36:18.212024 31574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:18.213212 31559 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 16:36:18.213343 31559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "25d712683b9b4eee8c48a04daa6676f6"
format_stamp: "Formatted at 2026-09-26 16:36:18 on dist-test-slave-2zvq"
server_key: "f36c4a7e538361e38e486ad676829c3f"
server_key_iv: "9e03a3c693426c86163037a6e1fbcf4b"
server_key_version: "encryptionkey@0"
I20260926 16:36:18.213466 31559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:18.236694 31559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:18.239854 31559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:18.240048 31559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:18.246860 31559 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35813
I20260926 16:36:18.246878 31626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35813 every 8 connection(s)
I20260926 16:36:18.247747 31559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:36:18.250444 31627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:18.251756 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 31559
I20260926 16:36:18.251900 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:36:18.252250 11299 external_mini_cluster.cc:1428] Setting key d946605479a94bc9a46240fc5ca8b615
I20260926 16:36:18.257246 31627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25d712683b9b4eee8c48a04daa6676f6: Bootstrap starting.
I20260926 16:36:18.259811 31627 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25d712683b9b4eee8c48a04daa6676f6: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:18.260545 31627 log.cc:824] T 00000000000000000000000000000000 P 25d712683b9b4eee8c48a04daa6676f6: Log is configured to *not* fsync() on all Append() calls
Sep 26 16:36:18 dist-test-slave-2zvq krb5kdc[31546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 16:36:18.262601 31627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25d712683b9b4eee8c48a04daa6676f6: No bootstrap required, opened a new log
I20260926 16:36:18.267079 31630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:18.253613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41023 (local address 127.0.0.1:35813)
0926 16:36:18.254010 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:18.254030 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:18.254072 (+ 42us) server_negotiation.cc:407] Connection header received
0926 16:36:18.254902 (+ 830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:18.254925 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:18.255342 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:18.255709 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:18.256383 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:18.257785 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:18.258699 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:18.259371 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:18.259526 (+ 155us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:18.261981 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:18.261988 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:18.262003 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:18.262031 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:18.264013 (+ 1982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:18.264498 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:18.264504 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:18.264510 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:18.264653 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:18.264960 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:18.264965 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:18.264968 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:18.265485 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:18.265695 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:18.265900 (+ 205us) server_negotiation.cc:299] Negotiation successful
0926 16:36:18.266114 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":96,"threads_started":1}
I20260926 16:36:18.267189 31627 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25d712683b9b4eee8c48a04daa6676f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25d712683b9b4eee8c48a04daa6676f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35813 } }
I20260926 16:36:18.267452 31627 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25d712683b9b4eee8c48a04daa6676f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:36:18.267506 31627 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25d712683b9b4eee8c48a04daa6676f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25d712683b9b4eee8c48a04daa6676f6, State: Initialized, Role: FOLLOWER
I20260926 16:36:18.267985 31627 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25d712683b9b4eee8c48a04daa6676f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25d712683b9b4eee8c48a04daa6676f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35813 } }
I20260926 16:36:18.268153 31627 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25d712683b9b4eee8c48a04daa6676f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:36:18.268219 31627 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25d712683b9b4eee8c48a04daa6676f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:36:18.268325 31627 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25d712683b9b4eee8c48a04daa6676f6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:18.269346 31627 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25d712683b9b4eee8c48a04daa6676f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25d712683b9b4eee8c48a04daa6676f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35813 } }
I20260926 16:36:18.269721 31627 leader_election.cc:302] T 00000000000000000000000000000000 P 25d712683b9b4eee8c48a04daa6676f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25d712683b9b4eee8c48a04daa6676f6; no voters:
I20260926 16:36:18.270056 31627 leader_election.cc:288] T 00000000000000000000000000000000 P 25d712683b9b4eee8c48a04daa6676f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:36:18.270089 11299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:36:18.270138 31632 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25d712683b9b4eee8c48a04daa6676f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:36:18.270361 31632 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25d712683b9b4eee8c48a04daa6676f6 [term 1 LEADER]: Becoming Leader. State: Replica: 25d712683b9b4eee8c48a04daa6676f6, State: Running, Role: LEADER
I20260926 16:36:18.270676 31632 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25d712683b9b4eee8c48a04daa6676f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25d712683b9b4eee8c48a04daa6676f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35813 } }
I20260926 16:36:18.271346 31627 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25d712683b9b4eee8c48a04daa6676f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:36:18.271709 31632 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25d712683b9b4eee8c48a04daa6676f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25d712683b9b4eee8c48a04daa6676f6. Latest consensus state: current_term: 1 leader_uuid: "25d712683b9b4eee8c48a04daa6676f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25d712683b9b4eee8c48a04daa6676f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35813 } } }
I20260926 16:36:18.271823 31632 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25d712683b9b4eee8c48a04daa6676f6 [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:18.272126 31633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25d712683b9b4eee8c48a04daa6676f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25d712683b9b4eee8c48a04daa6676f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25d712683b9b4eee8c48a04daa6676f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35813 } } }
I20260926 16:36:18.272204 31633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25d712683b9b4eee8c48a04daa6676f6 [sys.catalog]: This master's current role is: LEADER
W20260926 16:36:18.276647 31652 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 25d712683b9b4eee8c48a04daa6676f6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 16:36:18.276757 31652 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 16:36:18.276883 31640 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:36:18.277529 31640 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:36:18.281735 31630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:18.272336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47129 (local address 127.0.0.1:35813)
0926 16:36:18.272442 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:18.272446 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:18.272720 (+ 274us) server_negotiation.cc:407] Connection header received
0926 16:36:18.272907 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:18.272912 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:18.272969 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:18.273055 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:18.273999 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:18.274768 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:18.275749 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:18.276067 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:18.276169 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:18.277861 (+ 1692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:18.277865 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:18.277868 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:18.277892 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:18.279705 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:18.280252 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:18.280255 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:18.280258 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:18.280343 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:18.280699 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:18.280703 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:18.280704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:18.280962 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:18.281130 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:18.281530 (+ 400us) server_negotiation.cc:299] Negotiation successful
0926 16:36:18.281628 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260926 16:36:18.283474 31640 catalog_manager.cc:1347] Generated new cluster ID: 08af187918564cb08290b41692856fa7
I20260926 16:36:18.283532 31640 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:36:18.292415 31640 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:36:18.293529 31640 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:36:18.304911 31640 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25d712683b9b4eee8c48a04daa6676f6: Generated new TSK 0
I20260926 16:36:18.305541 31640 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:36:18.316135 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 31559
W20260926 16:36:18.321525 31544 connection.cc:535] client connection to 127.0.0.1:35813 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35813): Connection reset by peer (error 104)
2026-09-26T16:36:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1399 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:36:18 dist-test-slave-2zvq krb5kdc[31658](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:36:18 dist-test-slave-2zvq krb5kdc[31658](info): set up 2 sockets
Sep 26 16:36:18 dist-test-slave-2zvq krb5kdc[31658](info): commencing operation
krb5kdc: starting...
W20260926 16:36:19.340071 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.004s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:36:19 dist-test-slave-2zvq krb5kdc[31658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:36:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:36:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 16:36:19.456771 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46127
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46127
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:36:19.563980 31671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:19.564270 31671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:19.564358 31671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:19.568282 31671 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:36:19.568367 31671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:19.568408 31671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:19.568444 31671 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:36:19.568476 31671 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:36:19.573493 31671 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46127
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:19.574491 31671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:19.575474 31671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:19.583745 31676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:19.583832 31671 server_base.cc:1034] running on GCE node
W20260926 16:36:19.583750 31677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:19.583752 31679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:19.584528 31671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:19.585561 31671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:19.586726 31671 hybrid_clock.cc:648] HybridClock initialized: now 1790440579586714 us; error 25 us; skew 500 ppm
Sep 26 16:36:19 dist-test-slave-2zvq krb5kdc[31658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440579, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:36:19.589404 31671 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 16:36:19.590224 31671 webserver.cc:460] Webserver started at http://127.0.0.1:44843/ using document root <none> and password file <none>
I20260926 16:36:19.590811 31671 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:19.590878 31671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:19.591121 31671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:19.593075 31671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "e9efe964a47041a3a00f67f477bd5c54"
format_stamp: "Formatted at 2026-09-26 16:36:19 on dist-test-slave-2zvq"
server_key: "39c684db615cce81e90abba41062c37c"
server_key_iv: "7de33d028bbfb5f38c5765a6b21df224"
server_key_version: "encryptionkey@0"
I20260926 16:36:19.593624 31671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "e9efe964a47041a3a00f67f477bd5c54"
format_stamp: "Formatted at 2026-09-26 16:36:19 on dist-test-slave-2zvq"
server_key: "39c684db615cce81e90abba41062c37c"
server_key_iv: "7de33d028bbfb5f38c5765a6b21df224"
server_key_version: "encryptionkey@0"
I20260926 16:36:19.597216 31671 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260926 16:36:19.599850 31686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:19.601235 31671 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 16:36:19.601334 31671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "e9efe964a47041a3a00f67f477bd5c54"
format_stamp: "Formatted at 2026-09-26 16:36:19 on dist-test-slave-2zvq"
server_key: "39c684db615cce81e90abba41062c37c"
server_key_iv: "7de33d028bbfb5f38c5765a6b21df224"
server_key_version: "encryptionkey@0"
I20260926 16:36:19.601425 31671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:19.628824 31671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:19.632205 31671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:19.632398 31671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:19.639357 31671 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46127
I20260926 16:36:19.639384 31738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46127 every 8 connection(s)
I20260926 16:36:19.640285 31671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:36:19.642464 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 31671
I20260926 16:36:19.642601 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:36:19.642890 11299 external_mini_cluster.cc:1428] Setting key 13ecaef14b76e4abc320918e3a48e956
I20260926 16:36:19.643249 31739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:19.650492 31739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9efe964a47041a3a00f67f477bd5c54: Bootstrap starting.
I20260926 16:36:19.653016 31742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:19.644234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46321 (local address 127.0.0.1:46127)
0926 16:36:19.644673 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:19.644692 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:19.644734 (+ 42us) server_negotiation.cc:407] Connection header received
0926 16:36:19.645591 (+ 857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:19.645615 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:19.646047 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:19.646445 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:19.646863 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:19.648319 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:19.649254 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:19.649931 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:19.650085 (+ 154us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:19.651062 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:19.651068 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:19.651083 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:36:19.651556 (+ 473us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:19.651629 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:36:19.651664 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:19.651820 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:19.651991 (+ 171us) server_negotiation.cc:299] Negotiation successful
0926 16:36:19.652120 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":123,"threads_started":1}
I20260926 16:36:19.653230 31739 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e9efe964a47041a3a00f67f477bd5c54: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:19.654066 31739 log.cc:824] T 00000000000000000000000000000000 P e9efe964a47041a3a00f67f477bd5c54: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:19.656294 11299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:36:19.656316 31739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9efe964a47041a3a00f67f477bd5c54: No bootstrap required, opened a new log
I20260926 16:36:19.660030 31739 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9efe964a47041a3a00f67f477bd5c54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9efe964a47041a3a00f67f477bd5c54" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46127 } }
I20260926 16:36:19.660276 31739 raft_consensus.cc:374] T 00000000000000000000000000000000 P e9efe964a47041a3a00f67f477bd5c54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:36:19.660356 31739 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9efe964a47041a3a00f67f477bd5c54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9efe964a47041a3a00f67f477bd5c54, State: Initialized, Role: FOLLOWER
I20260926 16:36:19.660840 31739 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9efe964a47041a3a00f67f477bd5c54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9efe964a47041a3a00f67f477bd5c54" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46127 } }
I20260926 16:36:19.661005 31739 raft_consensus.cc:388] T 00000000000000000000000000000000 P e9efe964a47041a3a00f67f477bd5c54 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:36:19.661108 31739 raft_consensus.cc:482] T 00000000000000000000000000000000 P e9efe964a47041a3a00f67f477bd5c54 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:36:19.661218 31739 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9efe964a47041a3a00f67f477bd5c54 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:19.662623 31739 raft_consensus.cc:504] T 00000000000000000000000000000000 P e9efe964a47041a3a00f67f477bd5c54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9efe964a47041a3a00f67f477bd5c54" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46127 } }
I20260926 16:36:19.662745 31742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:19.658329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60423 (local address 127.0.0.1:46127)
0926 16:36:19.658444 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:19.658448 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:19.658621 (+ 173us) server_negotiation.cc:407] Connection header received
0926 16:36:19.658784 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:19.658788 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:19.658834 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:19.658915 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:19.659176 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:19.659695 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:19.660611 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:19.660948 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:19.661068 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:19.661997 (+ 929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:19.662000 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:19.662003 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:36:19.662015 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:19.662044 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:36:19.662055 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:19.662155 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:19.662509 (+ 354us) server_negotiation.cc:299] Negotiation successful
0926 16:36:19.662556 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 16:36:19.663023 31739 leader_election.cc:302] T 00000000000000000000000000000000 P e9efe964a47041a3a00f67f477bd5c54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e9efe964a47041a3a00f67f477bd5c54; no voters:
I20260926 16:36:19.663338 31739 leader_election.cc:288] T 00000000000000000000000000000000 P e9efe964a47041a3a00f67f477bd5c54 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:36:19.663439 31749 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e9efe964a47041a3a00f67f477bd5c54 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:36:19.663654 31749 raft_consensus.cc:686] T 00000000000000000000000000000000 P e9efe964a47041a3a00f67f477bd5c54 [term 1 LEADER]: Becoming Leader. State: Replica: e9efe964a47041a3a00f67f477bd5c54, State: Running, Role: LEADER
I20260926 16:36:19.664005 31749 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9efe964a47041a3a00f67f477bd5c54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9efe964a47041a3a00f67f477bd5c54" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46127 } }
I20260926 16:36:19.664592 31739 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9efe964a47041a3a00f67f477bd5c54 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:36:19.665585 31750 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9efe964a47041a3a00f67f477bd5c54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e9efe964a47041a3a00f67f477bd5c54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9efe964a47041a3a00f67f477bd5c54" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46127 } } }
I20260926 16:36:19.665719 31750 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9efe964a47041a3a00f67f477bd5c54 [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:19.666041 31751 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9efe964a47041a3a00f67f477bd5c54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9efe964a47041a3a00f67f477bd5c54. Latest consensus state: current_term: 1 leader_uuid: "e9efe964a47041a3a00f67f477bd5c54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9efe964a47041a3a00f67f477bd5c54" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46127 } } }
I20260926 16:36:19.666168 31751 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9efe964a47041a3a00f67f477bd5c54 [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:19.666100 31758 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:36:19.668951 31758 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:36:19.675418 31758 catalog_manager.cc:1347] Generated new cluster ID: bd08d5947eda44d7ace2e2cadbbb2855
I20260926 16:36:19.675493 31758 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:36:19.693670 31758 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:36:19.695053 31758 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:36:19.699687 31758 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e9efe964a47041a3a00f67f477bd5c54: Generated new TSK 0
I20260926 16:36:19.700485 31758 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:36:19.707621 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 31671
W20260926 16:36:19.712981 31656 connection.cc:535] client connection to 127.0.0.1:46127 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46127): Connection reset by peer (error 104)
2026-09-26T16:36:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1391 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:36:19 dist-test-slave-2zvq krb5kdc[31770](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:36:19 dist-test-slave-2zvq krb5kdc[31770](info): set up 2 sockets
Sep 26 16:36:19 dist-test-slave-2zvq krb5kdc[31770](info): commencing operation
krb5kdc: starting...
W20260926 16:36:20.764106 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:36:20 dist-test-slave-2zvq krb5kdc[31770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:36:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:36:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 16:36:20.879022 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40389
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40389
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:36:20.986937 31783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:20.987227 31783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:20.987306 31783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:20.991211 31783 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:36:20.991297 31783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:20.991348 31783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:20.991405 31783 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:36:20.991444 31783 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:36:20.996505 31783 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40389
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31783
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:20.997531 31783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:20.998555 31783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:21.007252 31789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:21.007263 31791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:21.007437 31783 server_base.cc:1034] running on GCE node
W20260926 16:36:21.007252 31788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:21.008153 31783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:21.009219 31783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:21.010371 31783 hybrid_clock.cc:648] HybridClock initialized: now 1790440581010344 us; error 45 us; skew 500 ppm
Sep 26 16:36:21 dist-test-slave-2zvq krb5kdc[31770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440581, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:36:21.013271 31783 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 16:36:21.014155 31783 webserver.cc:460] Webserver started at http://127.0.0.1:38485/ using document root <none> and password file <none>
I20260926 16:36:21.014731 31783 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:21.014801 31783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:21.014961 31783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:21.016921 31783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "265fc0f99c464055a9b9ad39a71a3034"
format_stamp: "Formatted at 2026-09-26 16:36:21 on dist-test-slave-2zvq"
server_key: "a2ff89f8c6f91b5a94b431e8e47e0b46"
server_key_iv: "77ca8d8a0b65c58dc0dcd29efc1d4803"
server_key_version: "encryptionkey@0"
I20260926 16:36:21.017416 31783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "265fc0f99c464055a9b9ad39a71a3034"
format_stamp: "Formatted at 2026-09-26 16:36:21 on dist-test-slave-2zvq"
server_key: "a2ff89f8c6f91b5a94b431e8e47e0b46"
server_key_iv: "77ca8d8a0b65c58dc0dcd29efc1d4803"
server_key_version: "encryptionkey@0"
I20260926 16:36:21.021055 31783 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260926 16:36:21.023530 31798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:21.024804 31783 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 16:36:21.024945 31783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "265fc0f99c464055a9b9ad39a71a3034"
format_stamp: "Formatted at 2026-09-26 16:36:21 on dist-test-slave-2zvq"
server_key: "a2ff89f8c6f91b5a94b431e8e47e0b46"
server_key_iv: "77ca8d8a0b65c58dc0dcd29efc1d4803"
server_key_version: "encryptionkey@0"
I20260926 16:36:21.025089 31783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:21.029655 31783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:21.032924 31783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:21.033113 31783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:21.040266 31783 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40389
I20260926 16:36:21.040290 31850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40389 every 8 connection(s)
I20260926 16:36:21.041136 31783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:36:21.043854 31851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:21.044200 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 31783
I20260926 16:36:21.044361 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:36:21.044653 11299 external_mini_cluster.cc:1428] Setting key 88d5a3d2ecd33170be9e1bc2ce54216c
I20260926 16:36:21.049767 31854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:21.046043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44951 (local address 127.0.0.1:40389)
0926 16:36:21.046473 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:21.046489 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:21.046522 (+ 33us) server_negotiation.cc:407] Connection header received
0926 16:36:21.047197 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:21.047214 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:21.047496 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:21.047766 (+ 270us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:21.048617 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:21.048622 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:21.048633 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:36:21.048644 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:21.048694 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:36:21.048712 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:21.048870 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:21.048945 (+ 75us) server_negotiation.cc:299] Negotiation successful
0926 16:36:21.049105 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":130,"threads_started":1}
I20260926 16:36:21.050621 31851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 265fc0f99c464055a9b9ad39a71a3034: Bootstrap starting.
I20260926 16:36:21.052423 11299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:36:21.052946 31851 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 265fc0f99c464055a9b9ad39a71a3034: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:21.053835 31851 log.cc:824] T 00000000000000000000000000000000 P 265fc0f99c464055a9b9ad39a71a3034: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:21.056181 31851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 265fc0f99c464055a9b9ad39a71a3034: No bootstrap required, opened a new log
I20260926 16:36:21.056315 31854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:21.054704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50197 (local address 127.0.0.1:40389)
0926 16:36:21.054831 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:21.054834 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:21.055071 (+ 237us) server_negotiation.cc:407] Connection header received
0926 16:36:21.055155 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:21.055157 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:21.055185 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:21.055253 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:21.055901 (+ 648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:21.055904 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:21.055906 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:36:21.055915 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:21.055938 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:36:21.055947 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:21.056016 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:21.056120 (+ 104us) server_negotiation.cc:299] Negotiation successful
0926 16:36:21.056149 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 16:36:21.059880 31851 raft_consensus.cc:348] T 00000000000000000000000000000000 P 265fc0f99c464055a9b9ad39a71a3034 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "265fc0f99c464055a9b9ad39a71a3034" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40389 } }
I20260926 16:36:21.060146 31851 raft_consensus.cc:374] T 00000000000000000000000000000000 P 265fc0f99c464055a9b9ad39a71a3034 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:36:21.060214 31851 raft_consensus.cc:729] T 00000000000000000000000000000000 P 265fc0f99c464055a9b9ad39a71a3034 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 265fc0f99c464055a9b9ad39a71a3034, State: Initialized, Role: FOLLOWER
I20260926 16:36:21.060583 31851 consensus_queue.cc:260] T 00000000000000000000000000000000 P 265fc0f99c464055a9b9ad39a71a3034 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "265fc0f99c464055a9b9ad39a71a3034" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40389 } }
I20260926 16:36:21.060699 31851 raft_consensus.cc:388] T 00000000000000000000000000000000 P 265fc0f99c464055a9b9ad39a71a3034 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:36:21.060745 31851 raft_consensus.cc:482] T 00000000000000000000000000000000 P 265fc0f99c464055a9b9ad39a71a3034 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:36:21.060812 31851 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 265fc0f99c464055a9b9ad39a71a3034 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:21.061662 31851 raft_consensus.cc:504] T 00000000000000000000000000000000 P 265fc0f99c464055a9b9ad39a71a3034 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "265fc0f99c464055a9b9ad39a71a3034" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40389 } }
I20260926 16:36:21.061944 31851 leader_election.cc:302] T 00000000000000000000000000000000 P 265fc0f99c464055a9b9ad39a71a3034 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 265fc0f99c464055a9b9ad39a71a3034; no voters:
I20260926 16:36:21.062184 31851 leader_election.cc:288] T 00000000000000000000000000000000 P 265fc0f99c464055a9b9ad39a71a3034 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:36:21.062331 31861 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 265fc0f99c464055a9b9ad39a71a3034 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:36:21.062585 31861 raft_consensus.cc:686] T 00000000000000000000000000000000 P 265fc0f99c464055a9b9ad39a71a3034 [term 1 LEADER]: Becoming Leader. State: Replica: 265fc0f99c464055a9b9ad39a71a3034, State: Running, Role: LEADER
I20260926 16:36:21.062916 31861 consensus_queue.cc:237] T 00000000000000000000000000000000 P 265fc0f99c464055a9b9ad39a71a3034 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "265fc0f99c464055a9b9ad39a71a3034" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40389 } }
I20260926 16:36:21.063408 31851 sys_catalog.cc:564] T 00000000000000000000000000000000 P 265fc0f99c464055a9b9ad39a71a3034 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:36:21.064492 31863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 265fc0f99c464055a9b9ad39a71a3034 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 265fc0f99c464055a9b9ad39a71a3034. Latest consensus state: current_term: 1 leader_uuid: "265fc0f99c464055a9b9ad39a71a3034" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "265fc0f99c464055a9b9ad39a71a3034" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40389 } } }
I20260926 16:36:21.064625 31863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 265fc0f99c464055a9b9ad39a71a3034 [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:21.065011 31870 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:36:21.064962 31862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 265fc0f99c464055a9b9ad39a71a3034 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "265fc0f99c464055a9b9ad39a71a3034" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "265fc0f99c464055a9b9ad39a71a3034" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40389 } } }
I20260926 16:36:21.065639 31862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 265fc0f99c464055a9b9ad39a71a3034 [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:21.068102 31870 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:36:21.073762 31870 catalog_manager.cc:1347] Generated new cluster ID: 27fa07a23521466b9fb1ad51673ca1ea
I20260926 16:36:21.073832 31870 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:36:21.088232 31870 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:36:21.088537 31870 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:36:21.096400 31870 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 265fc0f99c464055a9b9ad39a71a3034: Generated new TSK 0
I20260926 16:36:21.097014 31870 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:36:21.109901 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 31783
2026-09-26T16:36:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1402 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:36:21 dist-test-slave-2zvq krb5kdc[31882](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:36:21 dist-test-slave-2zvq krb5kdc[31882](info): set up 2 sockets
Sep 26 16:36:21 dist-test-slave-2zvq krb5kdc[31882](info): commencing operation
krb5kdc: starting...
W20260926 16:36:22.144234 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:36:22 dist-test-slave-2zvq krb5kdc[31882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:36:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:36:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 16:36:22.262605 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40961
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40961
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:36:22.368175 31895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:22.368521 31895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:22.368618 31895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:22.372570 31895 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:36:22.372622 31895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:22.372641 31895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:22.372658 31895 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:36:22.372673 31895 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:36:22.377665 31895 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40961
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31895
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:22.378633 31895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:22.379559 31895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:22.387663 31901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:22.387688 31903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:22.387717 31900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:22.387980 31895 server_base.cc:1034] running on GCE node
I20260926 16:36:22.388581 31895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:22.389614 31895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:22.390820 31895 hybrid_clock.cc:648] HybridClock initialized: now 1790440582390804 us; error 28 us; skew 500 ppm
Sep 26 16:36:22 dist-test-slave-2zvq krb5kdc[31882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440582, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:36:22.402321 31895 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 16:36:22.403235 31895 webserver.cc:460] Webserver started at http://127.0.0.1:36253/ using document root <none> and password file <none>
I20260926 16:36:22.403811 31895 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:22.403905 31895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:22.404139 31895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:22.406176 31895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "e701a4b2b4984306b57fb8b998c75eee"
format_stamp: "Formatted at 2026-09-26 16:36:22 on dist-test-slave-2zvq"
server_key: "815d16dac831466ee2486700d577e541"
server_key_iv: "1eaba4ec9c0a78422c4e4d463533568d"
server_key_version: "encryptionkey@0"
I20260926 16:36:22.406801 31895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "e701a4b2b4984306b57fb8b998c75eee"
format_stamp: "Formatted at 2026-09-26 16:36:22 on dist-test-slave-2zvq"
server_key: "815d16dac831466ee2486700d577e541"
server_key_iv: "1eaba4ec9c0a78422c4e4d463533568d"
server_key_version: "encryptionkey@0"
I20260926 16:36:22.410467 31895 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 16:36:22.412863 31910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:22.414129 31895 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:36:22.414348 31895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "e701a4b2b4984306b57fb8b998c75eee"
format_stamp: "Formatted at 2026-09-26 16:36:22 on dist-test-slave-2zvq"
server_key: "815d16dac831466ee2486700d577e541"
server_key_iv: "1eaba4ec9c0a78422c4e4d463533568d"
server_key_version: "encryptionkey@0"
I20260926 16:36:22.414471 31895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:22.436671 31895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:22.439740 31895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:22.439940 31895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:22.446926 31895 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40961
I20260926 16:36:22.446954 31962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40961 every 8 connection(s)
I20260926 16:36:22.447829 31895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:36:22.448091 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 31895
I20260926 16:36:22.448247 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:36:22.448566 11299 external_mini_cluster.cc:1428] Setting key ab773cf0e21b6c44c8624d2aff5dcf6b
I20260926 16:36:22.450887 31963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:22.457186 31963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e701a4b2b4984306b57fb8b998c75eee: Bootstrap starting.
Sep 26 16:36:22 dist-test-slave-2zvq krb5kdc[31882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 16:36:22.459800 31963 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e701a4b2b4984306b57fb8b998c75eee: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:22.460711 31963 log.cc:824] T 00000000000000000000000000000000 P e701a4b2b4984306b57fb8b998c75eee: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:22.462782 31963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e701a4b2b4984306b57fb8b998c75eee: No bootstrap required, opened a new log
I20260926 16:36:22.462970 31966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:22.450004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46073 (local address 127.0.0.1:40961)
0926 16:36:22.450461 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:22.450475 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:22.450509 (+ 34us) server_negotiation.cc:407] Connection header received
0926 16:36:22.451183 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:22.451200 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:22.451517 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:22.451782 (+ 265us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:22.452685 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:22.453666 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:22.454837 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:22.455289 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:22.455417 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:22.458127 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:22.458136 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:22.458154 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:22.458191 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:22.460096 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:22.460535 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:22.460541 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:22.460546 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:22.460677 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:22.460983 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:22.460987 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:22.460989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:22.461382 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:22.461585 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:22.461790 (+ 205us) server_negotiation.cc:299] Negotiation successful
0926 16:36:22.462068 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":137,"threads_started":1}
I20260926 16:36:22.465667 11299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:36:22.466693 31963 raft_consensus.cc:348] T 00000000000000000000000000000000 P e701a4b2b4984306b57fb8b998c75eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e701a4b2b4984306b57fb8b998c75eee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40961 } }
I20260926 16:36:22.466993 31963 raft_consensus.cc:374] T 00000000000000000000000000000000 P e701a4b2b4984306b57fb8b998c75eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:36:22.467083 31963 raft_consensus.cc:729] T 00000000000000000000000000000000 P e701a4b2b4984306b57fb8b998c75eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e701a4b2b4984306b57fb8b998c75eee, State: Initialized, Role: FOLLOWER
I20260926 16:36:22.467469 31963 consensus_queue.cc:260] T 00000000000000000000000000000000 P e701a4b2b4984306b57fb8b998c75eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e701a4b2b4984306b57fb8b998c75eee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40961 } }
I20260926 16:36:22.467620 31963 raft_consensus.cc:388] T 00000000000000000000000000000000 P e701a4b2b4984306b57fb8b998c75eee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:36:22.467694 31963 raft_consensus.cc:482] T 00000000000000000000000000000000 P e701a4b2b4984306b57fb8b998c75eee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:36:22.467793 31963 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e701a4b2b4984306b57fb8b998c75eee [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:22.468794 31963 raft_consensus.cc:504] T 00000000000000000000000000000000 P e701a4b2b4984306b57fb8b998c75eee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e701a4b2b4984306b57fb8b998c75eee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40961 } }
I20260926 16:36:22.469133 31963 leader_election.cc:302] T 00000000000000000000000000000000 P e701a4b2b4984306b57fb8b998c75eee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e701a4b2b4984306b57fb8b998c75eee; no voters:
I20260926 16:36:22.469445 31963 leader_election.cc:288] T 00000000000000000000000000000000 P e701a4b2b4984306b57fb8b998c75eee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:36:22.469610 31973 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e701a4b2b4984306b57fb8b998c75eee [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:36:22.469897 31973 raft_consensus.cc:686] T 00000000000000000000000000000000 P e701a4b2b4984306b57fb8b998c75eee [term 1 LEADER]: Becoming Leader. State: Replica: e701a4b2b4984306b57fb8b998c75eee, State: Running, Role: LEADER
I20260926 16:36:22.470283 31973 consensus_queue.cc:237] T 00000000000000000000000000000000 P e701a4b2b4984306b57fb8b998c75eee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e701a4b2b4984306b57fb8b998c75eee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40961 } }
I20260926 16:36:22.470674 31963 sys_catalog.cc:564] T 00000000000000000000000000000000 P e701a4b2b4984306b57fb8b998c75eee [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:36:22.472190 31974 sys_catalog.cc:455] T 00000000000000000000000000000000 P e701a4b2b4984306b57fb8b998c75eee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e701a4b2b4984306b57fb8b998c75eee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e701a4b2b4984306b57fb8b998c75eee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40961 } } }
I20260926 16:36:22.472302 31974 sys_catalog.cc:458] T 00000000000000000000000000000000 P e701a4b2b4984306b57fb8b998c75eee [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:22.473326 31975 sys_catalog.cc:455] T 00000000000000000000000000000000 P e701a4b2b4984306b57fb8b998c75eee [sys.catalog]: SysCatalogTable state changed. Reason: New leader e701a4b2b4984306b57fb8b998c75eee. Latest consensus state: current_term: 1 leader_uuid: "e701a4b2b4984306b57fb8b998c75eee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e701a4b2b4984306b57fb8b998c75eee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40961 } } }
I20260926 16:36:22.474237 31975 sys_catalog.cc:458] T 00000000000000000000000000000000 P e701a4b2b4984306b57fb8b998c75eee [sys.catalog]: This master's current role is: LEADER
W20260926 16:36:22.476095 31989 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e701a4b2b4984306b57fb8b998c75eee: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260926 16:36:22.476241 31966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:22.467685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36987 (local address 127.0.0.1:40961)
0926 16:36:22.467791 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:22.467795 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:22.467986 (+ 191us) server_negotiation.cc:407] Connection header received
0926 16:36:22.468059 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:22.468062 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:22.468108 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:22.468191 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:22.468759 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:22.469497 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:22.470656 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:22.471038 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:22.471137 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:22.472830 (+ 1693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:22.472837 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:22.472842 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:22.472868 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:22.474582 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:22.474997 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:22.475001 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:22.475003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:22.475081 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:22.475358 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:22.475362 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:22.475363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:22.475531 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:22.475666 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:22.475899 (+ 233us) server_negotiation.cc:299] Negotiation successful
0926 16:36:22.476017 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260926 16:36:22.476240 31989 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 16:36:22.476619 31985 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:36:22.477459 31985 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:36:22.483019 31985 catalog_manager.cc:1347] Generated new cluster ID: 4e2b51d13fc749bb9fcef8caa532a29e
I20260926 16:36:22.483081 31985 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:36:22.496424 31985 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:36:22.497524 31985 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:36:22.503002 31985 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e701a4b2b4984306b57fb8b998c75eee: Generated new TSK 0
I20260926 16:36:22.503662 31985 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:36:22.511215 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 31895
2026-09-26T16:36:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1400 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:36:22 dist-test-slave-2zvq krb5kdc[31994](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:36:22 dist-test-slave-2zvq krb5kdc[31994](info): set up 2 sockets
Sep 26 16:36:22 dist-test-slave-2zvq krb5kdc[31994](info): commencing operation
krb5kdc: starting...
W20260926 16:36:23.548278 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:36:23 dist-test-slave-2zvq krb5kdc[31994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:36:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:36:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 16:36:23.668853 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40801
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:36:23.776459 32007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:23.776762 32007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:23.776849 32007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:23.780870 32007 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:36:23.780959 32007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:23.780998 32007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:23.781035 32007 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:36:23.781067 32007 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:36:23.786311 32007 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:23.787365 32007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:23.788371 32007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:23.796919 32012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:23.796962 32013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:23.797001 32007 server_base.cc:1034] running on GCE node
W20260926 16:36:23.796936 32015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:23.797822 32007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:23.798966 32007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:23.800134 32007 hybrid_clock.cc:648] HybridClock initialized: now 1790440583800107 us; error 37 us; skew 500 ppm
Sep 26 16:36:23 dist-test-slave-2zvq krb5kdc[31994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440583, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:36:23.802898 32007 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 16:36:23.804029 32007 webserver.cc:460] Webserver started at http://127.0.0.1:40539/ using document root <none> and password file <none>
I20260926 16:36:23.804721 32007 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:23.804786 32007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:23.805011 32007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:23.807054 32007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "c3cb41494be94c29814b9feff0dd0ea8"
format_stamp: "Formatted at 2026-09-26 16:36:23 on dist-test-slave-2zvq"
server_key: "f898cd1b2eb545cd355534408e61074e"
server_key_iv: "668ece1033f1db5147908b2b2fcbc302"
server_key_version: "encryptionkey@0"
I20260926 16:36:23.807622 32007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "c3cb41494be94c29814b9feff0dd0ea8"
format_stamp: "Formatted at 2026-09-26 16:36:23 on dist-test-slave-2zvq"
server_key: "f898cd1b2eb545cd355534408e61074e"
server_key_iv: "668ece1033f1db5147908b2b2fcbc302"
server_key_version: "encryptionkey@0"
I20260926 16:36:23.811252 32007 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:36:23.813537 32022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:23.814815 32007 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:36:23.814942 32007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "c3cb41494be94c29814b9feff0dd0ea8"
format_stamp: "Formatted at 2026-09-26 16:36:23 on dist-test-slave-2zvq"
server_key: "f898cd1b2eb545cd355534408e61074e"
server_key_iv: "668ece1033f1db5147908b2b2fcbc302"
server_key_version: "encryptionkey@0"
I20260926 16:36:23.815040 32007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:23.843653 32007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:23.847208 32007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:23.847390 32007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:23.855213 32007 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40801
I20260926 16:36:23.855242 32074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40801 every 8 connection(s)
I20260926 16:36:23.856313 32007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:36:23.859185 32075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:23.864681 32075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3cb41494be94c29814b9feff0dd0ea8: Bootstrap starting.
I20260926 16:36:23.864935 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 32007
I20260926 16:36:23.865080 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:36:23.865352 11299 external_mini_cluster.cc:1428] Setting key d2b2e731049f6fe71f7f1e6aa44b2d64
I20260926 16:36:23.867398 32075 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3cb41494be94c29814b9feff0dd0ea8: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:23.868260 32075 log.cc:824] T 00000000000000000000000000000000 P c3cb41494be94c29814b9feff0dd0ea8: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:23.870433 32075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3cb41494be94c29814b9feff0dd0ea8: No bootstrap required, opened a new log
I20260926 16:36:23.874603 32075 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3cb41494be94c29814b9feff0dd0ea8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3cb41494be94c29814b9feff0dd0ea8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40801 } }
I20260926 16:36:23.874869 32075 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3cb41494be94c29814b9feff0dd0ea8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:36:23.874917 32075 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3cb41494be94c29814b9feff0dd0ea8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3cb41494be94c29814b9feff0dd0ea8, State: Initialized, Role: FOLLOWER
I20260926 16:36:23.875392 32075 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3cb41494be94c29814b9feff0dd0ea8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3cb41494be94c29814b9feff0dd0ea8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40801 } }
I20260926 16:36:23.875545 32075 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3cb41494be94c29814b9feff0dd0ea8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:36:23.875602 32078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:23.866837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37747 (local address 127.0.0.1:40801)
0926 16:36:23.867282 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:23.867303 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:23.867359 (+ 56us) server_negotiation.cc:407] Connection header received
0926 16:36:23.868215 (+ 856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:23.868238 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:23.868692 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:23.868984 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:23.869440 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:23.870869 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:23.871785 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:23.872503 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:23.872660 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:23.873475 (+ 815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:23.873481 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:23.873497 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:36:23.873997 (+ 500us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:23.874062 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:36:23.874091 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:23.874190 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:23.874429 (+ 239us) server_negotiation.cc:299] Negotiation successful
0926 16:36:23.874637 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":128,"threads_started":1}
I20260926 16:36:23.875613 32075 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3cb41494be94c29814b9feff0dd0ea8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:36:23.875730 32075 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3cb41494be94c29814b9feff0dd0ea8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:23.876775 32075 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3cb41494be94c29814b9feff0dd0ea8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3cb41494be94c29814b9feff0dd0ea8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40801 } }
I20260926 16:36:23.877113 32075 leader_election.cc:302] T 00000000000000000000000000000000 P c3cb41494be94c29814b9feff0dd0ea8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3cb41494be94c29814b9feff0dd0ea8; no voters:
I20260926 16:36:23.877434 32075 leader_election.cc:288] T 00000000000000000000000000000000 P c3cb41494be94c29814b9feff0dd0ea8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:36:23.877552 32080 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3cb41494be94c29814b9feff0dd0ea8 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:36:23.877817 32080 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3cb41494be94c29814b9feff0dd0ea8 [term 1 LEADER]: Becoming Leader. State: Replica: c3cb41494be94c29814b9feff0dd0ea8, State: Running, Role: LEADER
I20260926 16:36:23.878173 32080 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3cb41494be94c29814b9feff0dd0ea8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3cb41494be94c29814b9feff0dd0ea8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40801 } }
I20260926 16:36:23.878538 11299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:36:23.878791 32075 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3cb41494be94c29814b9feff0dd0ea8 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:36:23.879992 32081 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3cb41494be94c29814b9feff0dd0ea8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3cb41494be94c29814b9feff0dd0ea8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3cb41494be94c29814b9feff0dd0ea8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40801 } } }
I20260926 16:36:23.880134 32081 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3cb41494be94c29814b9feff0dd0ea8 [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:23.880510 32093 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:36:23.881294 32082 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3cb41494be94c29814b9feff0dd0ea8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3cb41494be94c29814b9feff0dd0ea8. Latest consensus state: current_term: 1 leader_uuid: "c3cb41494be94c29814b9feff0dd0ea8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3cb41494be94c29814b9feff0dd0ea8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40801 } } }
I20260926 16:36:23.881506 32082 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3cb41494be94c29814b9feff0dd0ea8 [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:23.883823 32093 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:36:23.885648 32078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:23.881036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57415 (local address 127.0.0.1:40801)
0926 16:36:23.881136 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:23.881139 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:23.881372 (+ 233us) server_negotiation.cc:407] Connection header received
0926 16:36:23.881554 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:23.881558 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:23.881600 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:23.881647 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:23.882041 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:23.882797 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:23.883916 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:23.884215 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:23.884322 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:23.885180 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:23.885184 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:23.885187 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:36:23.885199 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:23.885229 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:36:23.885244 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:23.885290 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:23.885475 (+ 185us) server_negotiation.cc:299] Negotiation successful
0926 16:36:23.885526 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 16:36:23.889492 32093 catalog_manager.cc:1347] Generated new cluster ID: ab2a7f90325241d8a1e4382b66066172
I20260926 16:36:23.889555 32093 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:36:23.940420 32093 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:36:23.941713 32093 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:36:23.948722 32093 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3cb41494be94c29814b9feff0dd0ea8: Generated new TSK 0
I20260926 16:36:23.949507 32093 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:36:23.959617 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 32007
2026-09-26T16:36:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1450 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:36:24 dist-test-slave-2zvq krb5kdc[32106](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:36:24 dist-test-slave-2zvq krb5kdc[32106](info): set up 2 sockets
Sep 26 16:36:24 dist-test-slave-2zvq krb5kdc[32106](info): commencing operation
krb5kdc: starting...
W20260926 16:36:24.988088 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:36:25 dist-test-slave-2zvq krb5kdc[32106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:36:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:36:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 16:36:25.106148 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43589
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43589
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:36:25.211418 32119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:25.211709 32119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:25.211784 32119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:25.215750 32119 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:36:25.215843 32119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:25.215884 32119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:25.215940 32119 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:36:25.215981 32119 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:36:25.221033 32119 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43589
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:25.222021 32119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:25.223023 32119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:25.231442 32125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:25.231483 32127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:25.231458 32124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:25.231714 32119 server_base.cc:1034] running on GCE node
I20260926 16:36:25.232381 32119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:25.233415 32119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:25.234578 32119 hybrid_clock.cc:648] HybridClock initialized: now 1790440585234561 us; error 29 us; skew 500 ppm
Sep 26 16:36:25 dist-test-slave-2zvq krb5kdc[32106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440585, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:36:25.237222 32119 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 16:36:25.238078 32119 webserver.cc:460] Webserver started at http://127.0.0.1:44889/ using document root <none> and password file <none>
I20260926 16:36:25.238641 32119 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:25.238701 32119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:25.238866 32119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:25.240772 32119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "ea8d1445f0384870a8afd1ee1ea57853"
format_stamp: "Formatted at 2026-09-26 16:36:25 on dist-test-slave-2zvq"
server_key: "e20e3942be6689f7ee7267f7a7296d61"
server_key_iv: "2f80cbcd73bda41ce440cbb49fcee0d9"
server_key_version: "encryptionkey@0"
I20260926 16:36:25.241282 32119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "ea8d1445f0384870a8afd1ee1ea57853"
format_stamp: "Formatted at 2026-09-26 16:36:25 on dist-test-slave-2zvq"
server_key: "e20e3942be6689f7ee7267f7a7296d61"
server_key_iv: "2f80cbcd73bda41ce440cbb49fcee0d9"
server_key_version: "encryptionkey@0"
I20260926 16:36:25.244772 32119 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260926 16:36:25.247097 32134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:25.248363 32119 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:36:25.248504 32119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "ea8d1445f0384870a8afd1ee1ea57853"
format_stamp: "Formatted at 2026-09-26 16:36:25 on dist-test-slave-2zvq"
server_key: "e20e3942be6689f7ee7267f7a7296d61"
server_key_iv: "2f80cbcd73bda41ce440cbb49fcee0d9"
server_key_version: "encryptionkey@0"
I20260926 16:36:25.248611 32119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:25.252982 32119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:25.256354 32119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:25.256551 32119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:25.263865 32119 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43589
I20260926 16:36:25.263919 32186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43589 every 8 connection(s)
I20260926 16:36:25.264776 32119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:36:25.267478 32187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:25.271272 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 32119
I20260926 16:36:25.271447 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:36:25.271710 11299 external_mini_cluster.cc:1428] Setting key c8241368944ca3ddc4584ddd8d03474b
I20260926 16:36:25.273026 32187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea8d1445f0384870a8afd1ee1ea57853: Bootstrap starting.
I20260926 16:36:25.275305 32187 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea8d1445f0384870a8afd1ee1ea57853: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:25.276279 32187 log.cc:824] T 00000000000000000000000000000000 P ea8d1445f0384870a8afd1ee1ea57853: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:25.277086 32190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:25.273162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54589 (local address 127.0.0.1:43589)
0926 16:36:25.273589 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:25.273604 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:25.273638 (+ 34us) server_negotiation.cc:407] Connection header received
0926 16:36:25.274252 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:25.274327 (+ 75us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:25.274616 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:25.274891 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:25.275721 (+ 830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:25.275726 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:25.275737 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:36:25.275747 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:25.275798 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:36:25.275816 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:25.275897 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:25.276034 (+ 137us) server_negotiation.cc:299] Negotiation successful
0926 16:36:25.276198 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":130,"threads_started":1}
I20260926 16:36:25.278206 32187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea8d1445f0384870a8afd1ee1ea57853: No bootstrap required, opened a new log
I20260926 16:36:25.279907 11299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:36:25.282213 32187 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea8d1445f0384870a8afd1ee1ea57853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea8d1445f0384870a8afd1ee1ea57853" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43589 } }
I20260926 16:36:25.282521 32187 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea8d1445f0384870a8afd1ee1ea57853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:36:25.282584 32187 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea8d1445f0384870a8afd1ee1ea57853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea8d1445f0384870a8afd1ee1ea57853, State: Initialized, Role: FOLLOWER
I20260926 16:36:25.283041 32187 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea8d1445f0384870a8afd1ee1ea57853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea8d1445f0384870a8afd1ee1ea57853" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43589 } }
I20260926 16:36:25.283161 32187 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea8d1445f0384870a8afd1ee1ea57853 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:36:25.283241 32187 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea8d1445f0384870a8afd1ee1ea57853 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:36:25.283356 32187 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea8d1445f0384870a8afd1ee1ea57853 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:25.283608 32190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:25.281852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37441 (local address 127.0.0.1:43589)
0926 16:36:25.281982 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:25.281986 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:25.282197 (+ 211us) server_negotiation.cc:407] Connection header received
0926 16:36:25.282374 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:25.282376 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:25.282404 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:25.282467 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:25.283248 (+ 781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:25.283251 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:25.283254 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:36:25.283264 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:25.283291 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:36:25.283306 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:25.283395 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:25.283468 (+ 73us) server_negotiation.cc:299] Negotiation successful
0926 16:36:25.283504 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 16:36:25.284441 32187 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea8d1445f0384870a8afd1ee1ea57853 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea8d1445f0384870a8afd1ee1ea57853" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43589 } }
I20260926 16:36:25.284775 32187 leader_election.cc:302] T 00000000000000000000000000000000 P ea8d1445f0384870a8afd1ee1ea57853 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea8d1445f0384870a8afd1ee1ea57853; no voters:
I20260926 16:36:25.285049 32187 leader_election.cc:288] T 00000000000000000000000000000000 P ea8d1445f0384870a8afd1ee1ea57853 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:36:25.285127 32197 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea8d1445f0384870a8afd1ee1ea57853 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:36:25.285383 32197 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea8d1445f0384870a8afd1ee1ea57853 [term 1 LEADER]: Becoming Leader. State: Replica: ea8d1445f0384870a8afd1ee1ea57853, State: Running, Role: LEADER
I20260926 16:36:25.285713 32197 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea8d1445f0384870a8afd1ee1ea57853 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea8d1445f0384870a8afd1ee1ea57853" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43589 } }
I20260926 16:36:25.286207 32187 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea8d1445f0384870a8afd1ee1ea57853 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:36:25.287487 32199 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea8d1445f0384870a8afd1ee1ea57853 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea8d1445f0384870a8afd1ee1ea57853. Latest consensus state: current_term: 1 leader_uuid: "ea8d1445f0384870a8afd1ee1ea57853" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea8d1445f0384870a8afd1ee1ea57853" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43589 } } }
I20260926 16:36:25.287468 32198 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea8d1445f0384870a8afd1ee1ea57853 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea8d1445f0384870a8afd1ee1ea57853" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea8d1445f0384870a8afd1ee1ea57853" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43589 } } }
I20260926 16:36:25.287635 32199 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea8d1445f0384870a8afd1ee1ea57853 [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:25.287639 32198 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea8d1445f0384870a8afd1ee1ea57853 [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:25.288419 32207 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:36:25.291047 32207 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:36:25.296350 32207 catalog_manager.cc:1347] Generated new cluster ID: a3de5a00823b423d8e9a91ce63e35f21
I20260926 16:36:25.296435 32207 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:36:25.306980 32207 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:36:25.307305 32207 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:36:25.311774 32207 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea8d1445f0384870a8afd1ee1ea57853: Generated new TSK 0
I20260926 16:36:25.312454 32207 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:36:25.321298 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 32119
2026-09-26T16:36:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1360 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:36:25 dist-test-slave-2zvq krb5kdc[32218](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:36:25 dist-test-slave-2zvq krb5kdc[32218](info): set up 2 sockets
Sep 26 16:36:25 dist-test-slave-2zvq krb5kdc[32218](info): commencing operation
krb5kdc: starting...
W20260926 16:36:26.364363 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:36:26 dist-test-slave-2zvq krb5kdc[32218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:36:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:36:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 16:36:26.485037 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39113
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39113
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:36:26.591193 32231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:26.591449 32231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:26.591490 32231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:26.595317 32231 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:36:26.595366 32231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:26.595384 32231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:26.595402 32231 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:36:26.595417 32231 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:36:26.600482 32231 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39113
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:26.601464 32231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:26.602438 32231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:26.610800 32236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:26.610942 32231 server_base.cc:1034] running on GCE node
W20260926 16:36:26.610785 32239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:26.610726 32237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:26.611645 32231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:26.612761 32231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:26.613961 32231 hybrid_clock.cc:648] HybridClock initialized: now 1790440586613920 us; error 51 us; skew 500 ppm
Sep 26 16:36:26 dist-test-slave-2zvq krb5kdc[32218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440586, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:36:26.616861 32231 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 16:36:26.617728 32231 webserver.cc:460] Webserver started at http://127.0.0.1:39361/ using document root <none> and password file <none>
I20260926 16:36:26.618336 32231 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:26.618402 32231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:26.618566 32231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:26.620600 32231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "546cd5bdc9a140b1a93940ad3e4cd412"
format_stamp: "Formatted at 2026-09-26 16:36:26 on dist-test-slave-2zvq"
server_key: "b7e2eacaf1d82a1984f39dd23a485a0f"
server_key_iv: "46d064b313bce3e3632e6950e7ba8b64"
server_key_version: "encryptionkey@0"
I20260926 16:36:26.621102 32231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "546cd5bdc9a140b1a93940ad3e4cd412"
format_stamp: "Formatted at 2026-09-26 16:36:26 on dist-test-slave-2zvq"
server_key: "b7e2eacaf1d82a1984f39dd23a485a0f"
server_key_iv: "46d064b313bce3e3632e6950e7ba8b64"
server_key_version: "encryptionkey@0"
I20260926 16:36:26.624692 32231 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.001s
I20260926 16:36:26.627043 32246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:26.628199 32231 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260926 16:36:26.628329 32231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "546cd5bdc9a140b1a93940ad3e4cd412"
format_stamp: "Formatted at 2026-09-26 16:36:26 on dist-test-slave-2zvq"
server_key: "b7e2eacaf1d82a1984f39dd23a485a0f"
server_key_iv: "46d064b313bce3e3632e6950e7ba8b64"
server_key_version: "encryptionkey@0"
I20260926 16:36:26.628432 32231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:26.666299 32231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:26.669631 32231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:26.669836 32231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:26.676713 32231 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39113
I20260926 16:36:26.676731 32298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39113 every 8 connection(s)
I20260926 16:36:26.677609 32231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:36:26.680264 32299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:26.680907 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 32231
I20260926 16:36:26.681034 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:36:26.681324 11299 external_mini_cluster.cc:1428] Setting key 9dc8c0e0dbf20033aed9b7f810627025
I20260926 16:36:26.686849 32299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 546cd5bdc9a140b1a93940ad3e4cd412: Bootstrap starting.
I20260926 16:36:26.689245 32299 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 546cd5bdc9a140b1a93940ad3e4cd412: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:26.689949 32299 log.cc:824] T 00000000000000000000000000000000 P 546cd5bdc9a140b1a93940ad3e4cd412: Log is configured to *not* fsync() on all Append() calls
Sep 26 16:36:26 dist-test-slave-2zvq krb5kdc[32218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 16:36:26.692019 32299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 546cd5bdc9a140b1a93940ad3e4cd412: No bootstrap required, opened a new log
I20260926 16:36:26.696426 32302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:26.682787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.171:38091 (local address 127.0.0.1:39113)
0926 16:36:26.683217 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:26.683238 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:26.683279 (+ 41us) server_negotiation.cc:407] Connection header received
0926 16:36:26.684105 (+ 826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:26.684131 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:26.684558 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:26.684914 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:26.685556 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:26.687017 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:26.687893 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:26.688563 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:26.688714 (+ 151us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:26.691293 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:26.691299 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:26.691314 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:26.691351 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:26.693263 (+ 1912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:26.693747 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:26.693753 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:26.693758 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:26.693902 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:26.694311 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:26.694314 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:26.694316 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:26.694807 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:26.694998 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:26.695281 (+ 283us) server_negotiation.cc:299] Negotiation successful
0926 16:36:26.695505 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":112,"threads_started":1}
I20260926 16:36:26.696468 32299 raft_consensus.cc:348] T 00000000000000000000000000000000 P 546cd5bdc9a140b1a93940ad3e4cd412 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "546cd5bdc9a140b1a93940ad3e4cd412" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39113 } }
I20260926 16:36:26.696700 32299 raft_consensus.cc:374] T 00000000000000000000000000000000 P 546cd5bdc9a140b1a93940ad3e4cd412 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:36:26.696746 32299 raft_consensus.cc:729] T 00000000000000000000000000000000 P 546cd5bdc9a140b1a93940ad3e4cd412 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 546cd5bdc9a140b1a93940ad3e4cd412, State: Initialized, Role: FOLLOWER
I20260926 16:36:26.697167 32299 consensus_queue.cc:260] T 00000000000000000000000000000000 P 546cd5bdc9a140b1a93940ad3e4cd412 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "546cd5bdc9a140b1a93940ad3e4cd412" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39113 } }
I20260926 16:36:26.697273 32299 raft_consensus.cc:388] T 00000000000000000000000000000000 P 546cd5bdc9a140b1a93940ad3e4cd412 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:36:26.697310 32299 raft_consensus.cc:482] T 00000000000000000000000000000000 P 546cd5bdc9a140b1a93940ad3e4cd412 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:36:26.697373 32299 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 546cd5bdc9a140b1a93940ad3e4cd412 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:26.698181 32299 raft_consensus.cc:504] T 00000000000000000000000000000000 P 546cd5bdc9a140b1a93940ad3e4cd412 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "546cd5bdc9a140b1a93940ad3e4cd412" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39113 } }
I20260926 16:36:26.698554 32299 leader_election.cc:302] T 00000000000000000000000000000000 P 546cd5bdc9a140b1a93940ad3e4cd412 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 546cd5bdc9a140b1a93940ad3e4cd412; no voters:
I20260926 16:36:26.698845 32299 leader_election.cc:288] T 00000000000000000000000000000000 P 546cd5bdc9a140b1a93940ad3e4cd412 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:36:26.698932 32304 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 546cd5bdc9a140b1a93940ad3e4cd412 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:36:26.699162 32304 raft_consensus.cc:686] T 00000000000000000000000000000000 P 546cd5bdc9a140b1a93940ad3e4cd412 [term 1 LEADER]: Becoming Leader. State: Replica: 546cd5bdc9a140b1a93940ad3e4cd412, State: Running, Role: LEADER
I20260926 16:36:26.699451 11299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:36:26.699476 32304 consensus_queue.cc:237] T 00000000000000000000000000000000 P 546cd5bdc9a140b1a93940ad3e4cd412 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "546cd5bdc9a140b1a93940ad3e4cd412" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39113 } }
I20260926 16:36:26.700090 32299 sys_catalog.cc:564] T 00000000000000000000000000000000 P 546cd5bdc9a140b1a93940ad3e4cd412 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:36:26.700632 32304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 546cd5bdc9a140b1a93940ad3e4cd412 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "546cd5bdc9a140b1a93940ad3e4cd412" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "546cd5bdc9a140b1a93940ad3e4cd412" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39113 } } }
I20260926 16:36:26.700754 32304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 546cd5bdc9a140b1a93940ad3e4cd412 [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:26.701049 32305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 546cd5bdc9a140b1a93940ad3e4cd412 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 546cd5bdc9a140b1a93940ad3e4cd412. Latest consensus state: current_term: 1 leader_uuid: "546cd5bdc9a140b1a93940ad3e4cd412" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "546cd5bdc9a140b1a93940ad3e4cd412" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39113 } } }
I20260926 16:36:26.701165 32305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 546cd5bdc9a140b1a93940ad3e4cd412 [sys.catalog]: This master's current role is: LEADER
W20260926 16:36:26.704941 32324 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 546cd5bdc9a140b1a93940ad3e4cd412: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 16:36:26.705021 32324 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 16:36:26.705133 32312 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:36:26.706000 32312 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:36:26.710656 32302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:26.701688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.171:56571 (local address 127.0.0.1:39113)
0926 16:36:26.701792 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:26.701796 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:26.702111 (+ 315us) server_negotiation.cc:407] Connection header received
0926 16:36:26.702264 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:26.702267 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:26.702329 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:26.702381 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:26.703337 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:26.704103 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:26.705218 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:26.705466 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:26.705577 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:26.707023 (+ 1446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:26.707027 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:26.707029 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:26.707053 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:26.708694 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:26.709218 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:26.709222 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:26.709224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:26.709313 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:26.709668 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:26.709671 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:26.709673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:26.709886 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:26.710034 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:26.710391 (+ 357us) server_negotiation.cc:299] Negotiation successful
0926 16:36:26.710478 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 16:36:26.715484 32312 catalog_manager.cc:1347] Generated new cluster ID: 1725a5961127466691243094e1e0c90b
I20260926 16:36:26.715546 32312 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:36:26.729203 32312 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:36:26.730273 32312 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:36:26.740300 32312 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 546cd5bdc9a140b1a93940ad3e4cd412: Generated new TSK 0
I20260926 16:36:26.741111 32312 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:36:26.747347 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 32231
W20260926 16:36:26.753334 32216 connection.cc:535] client connection to 127.0.0.1:39113 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39113): Connection reset by peer (error 104)
2026-09-26T16:36:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1426 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:36:26 dist-test-slave-2zvq krb5kdc[32330](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:36:26 dist-test-slave-2zvq krb5kdc[32330](info): set up 2 sockets
Sep 26 16:36:26 dist-test-slave-2zvq krb5kdc[32330](info): commencing operation
krb5kdc: starting...
W20260926 16:36:27.840173 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:36:27 dist-test-slave-2zvq krb5kdc[32330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:36:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:36:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 16:36:27.957935 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35423
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35423
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:36:28.063861 32343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:28.064138 32343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:28.064213 32343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:28.068291 32343 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:36:28.068369 32343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:28.068408 32343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:28.068461 32343 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:36:28.068500 32343 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:36:28.074038 32343 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35423
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:28.075037 32343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:28.076038 32343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:28.084316 32348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:28.084571 32343 server_base.cc:1034] running on GCE node
W20260926 16:36:28.084337 32349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:28.084292 32351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:28.085328 32343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:28.086475 32343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:28.087669 32343 hybrid_clock.cc:648] HybridClock initialized: now 1790440588087651 us; error 30 us; skew 500 ppm
Sep 26 16:36:28 dist-test-slave-2zvq krb5kdc[32330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440588, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:36:28.100236 32343 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 16:36:28.101516 32343 webserver.cc:460] Webserver started at http://127.0.0.1:40729/ using document root <none> and password file <none>
I20260926 16:36:28.102353 32343 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:28.102419 32343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:28.102690 32343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:28.104758 32343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "7d92d23451824e03b09c35d0ae46c1a4"
format_stamp: "Formatted at 2026-09-26 16:36:28 on dist-test-slave-2zvq"
server_key: "74baee40d1dcb0da3ecccc3a24181b73"
server_key_iv: "bf1f6a3be4547fb2216be6c016322723"
server_key_version: "encryptionkey@0"
I20260926 16:36:28.105333 32343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "7d92d23451824e03b09c35d0ae46c1a4"
format_stamp: "Formatted at 2026-09-26 16:36:28 on dist-test-slave-2zvq"
server_key: "74baee40d1dcb0da3ecccc3a24181b73"
server_key_iv: "bf1f6a3be4547fb2216be6c016322723"
server_key_version: "encryptionkey@0"
I20260926 16:36:28.109002 32343 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260926 16:36:28.111431 32358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:28.112632 32343 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 16:36:28.112787 32343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "7d92d23451824e03b09c35d0ae46c1a4"
format_stamp: "Formatted at 2026-09-26 16:36:28 on dist-test-slave-2zvq"
server_key: "74baee40d1dcb0da3ecccc3a24181b73"
server_key_iv: "bf1f6a3be4547fb2216be6c016322723"
server_key_version: "encryptionkey@0"
I20260926 16:36:28.112900 32343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:28.125532 32343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:28.128623 32343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:28.128818 32343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:28.135934 32343 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35423
I20260926 16:36:28.135957 32410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35423 every 8 connection(s)
I20260926 16:36:28.136854 32343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:36:28.139626 32411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:28.143566 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 32343
I20260926 16:36:28.143700 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:36:28.143970 11299 external_mini_cluster.cc:1428] Setting key 5e90c46afbf69af014e6e6100e323159
I20260926 16:36:28.145391 32411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d92d23451824e03b09c35d0ae46c1a4: Bootstrap starting.
I20260926 16:36:28.147598 32411 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d92d23451824e03b09c35d0ae46c1a4: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:28.148310 32411 log.cc:824] T 00000000000000000000000000000000 P 7d92d23451824e03b09c35d0ae46c1a4: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:28.150032 32411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d92d23451824e03b09c35d0ae46c1a4: No bootstrap required, opened a new log
I20260926 16:36:28.152670 32414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:28.145392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.171:41913 (local address 127.0.0.1:35423)
0926 16:36:28.145828 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:28.145843 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:28.145879 (+ 36us) server_negotiation.cc:407] Connection header received
0926 16:36:28.146529 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:28.146547 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:28.146867 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:28.147207 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:28.147642 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:28.148707 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:28.149581 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:28.150033 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:28.150217 (+ 184us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:28.151116 (+ 899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:28.151121 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:28.151133 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:36:28.151503 (+ 370us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:28.151552 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:36:28.151573 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:28.151709 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:28.151894 (+ 185us) server_negotiation.cc:299] Negotiation successful
0926 16:36:28.152024 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":129,"threads_started":1}
I20260926 16:36:28.153573 32411 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d92d23451824e03b09c35d0ae46c1a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d92d23451824e03b09c35d0ae46c1a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35423 } }
I20260926 16:36:28.153800 32411 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d92d23451824e03b09c35d0ae46c1a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:36:28.153891 32411 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d92d23451824e03b09c35d0ae46c1a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d92d23451824e03b09c35d0ae46c1a4, State: Initialized, Role: FOLLOWER
I20260926 16:36:28.154368 32411 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d92d23451824e03b09c35d0ae46c1a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d92d23451824e03b09c35d0ae46c1a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35423 } }
I20260926 16:36:28.154538 32411 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d92d23451824e03b09c35d0ae46c1a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:36:28.154606 32411 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d92d23451824e03b09c35d0ae46c1a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:36:28.154700 32411 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d92d23451824e03b09c35d0ae46c1a4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:28.155570 11299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:36:28.155787 32411 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d92d23451824e03b09c35d0ae46c1a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d92d23451824e03b09c35d0ae46c1a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35423 } }
I20260926 16:36:28.156111 32411 leader_election.cc:302] T 00000000000000000000000000000000 P 7d92d23451824e03b09c35d0ae46c1a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d92d23451824e03b09c35d0ae46c1a4; no voters:
I20260926 16:36:28.156414 32411 leader_election.cc:288] T 00000000000000000000000000000000 P 7d92d23451824e03b09c35d0ae46c1a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:36:28.156515 32416 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d92d23451824e03b09c35d0ae46c1a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:36:28.156714 32416 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d92d23451824e03b09c35d0ae46c1a4 [term 1 LEADER]: Becoming Leader. State: Replica: 7d92d23451824e03b09c35d0ae46c1a4, State: Running, Role: LEADER
I20260926 16:36:28.157058 32416 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d92d23451824e03b09c35d0ae46c1a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d92d23451824e03b09c35d0ae46c1a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35423 } }
I20260926 16:36:28.157697 32411 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d92d23451824e03b09c35d0ae46c1a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:36:28.158239 32418 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d92d23451824e03b09c35d0ae46c1a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d92d23451824e03b09c35d0ae46c1a4. Latest consensus state: current_term: 1 leader_uuid: "7d92d23451824e03b09c35d0ae46c1a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d92d23451824e03b09c35d0ae46c1a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35423 } } }
I20260926 16:36:28.158388 32418 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d92d23451824e03b09c35d0ae46c1a4 [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:28.158811 32417 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d92d23451824e03b09c35d0ae46c1a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d92d23451824e03b09c35d0ae46c1a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d92d23451824e03b09c35d0ae46c1a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35423 } } }
I20260926 16:36:28.158943 32417 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d92d23451824e03b09c35d0ae46c1a4 [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:28.158815 32427 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:36:28.161933 32427 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:36:28.162882 32414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:28.157678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.171:57719 (local address 127.0.0.1:35423)
0926 16:36:28.157769 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:28.157773 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:28.158233 (+ 460us) server_negotiation.cc:407] Connection header received
0926 16:36:28.158447 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:28.158451 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:28.158496 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:28.158551 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:28.158978 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:28.159729 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:28.160908 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:28.161234 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:28.161354 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:28.162215 (+ 861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:28.162220 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:28.162226 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:36:28.162241 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:28.162287 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:36:28.162302 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:28.162385 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:28.162729 (+ 344us) server_negotiation.cc:299] Negotiation successful
0926 16:36:28.162766 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260926 16:36:28.167629 32427 catalog_manager.cc:1347] Generated new cluster ID: 3cfd8ff216c240b2854079696b65a70e
I20260926 16:36:28.167712 32427 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:36:28.191738 32427 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:36:28.192724 32427 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:36:28.200824 32427 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d92d23451824e03b09c35d0ae46c1a4: Generated new TSK 0
I20260926 16:36:28.201524 32427 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:36:28.211051 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 32343
W20260926 16:36:28.216404 32328 connection.cc:535] client connection to 127.0.0.1:35423 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35423): Connection reset by peer (error 104)
2026-09-26T16:36:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1462 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:36:28 dist-test-slave-2zvq krb5kdc[32442](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:36:28 dist-test-slave-2zvq krb5kdc[32442](info): set up 2 sockets
Sep 26 16:36:28 dist-test-slave-2zvq krb5kdc[32442](info): commencing operation
krb5kdc: starting...
W20260926 16:36:29.248258 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:36:29 dist-test-slave-2zvq krb5kdc[32442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:36:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:36:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 16:36:29.363198 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36235
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36235
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:36:29.470917 32455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:29.471153 32455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:29.471220 32455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:29.475101 32455 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:36:29.475154 32455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:29.475173 32455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:29.475189 32455 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:36:29.475247 32455 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:36:29.480373 32455 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36235
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:29.481356 32455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:29.482347 32455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:29.490671 32460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:29.490664 32463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:29.490664 32461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:29.490732 32455 server_base.cc:1034] running on GCE node
I20260926 16:36:29.491374 32455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:29.492412 32455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:29.493600 32455 hybrid_clock.cc:648] HybridClock initialized: now 1790440589493575 us; error 44 us; skew 500 ppm
Sep 26 16:36:29 dist-test-slave-2zvq krb5kdc[32442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440589, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:36:29.496276 32455 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 16:36:29.497092 32455 webserver.cc:460] Webserver started at http://127.0.0.1:37305/ using document root <none> and password file <none>
I20260926 16:36:29.497627 32455 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:29.497713 32455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:29.497946 32455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:29.499965 32455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "ae09c4534b6f47ebaa871e6ec8f6a4ca"
format_stamp: "Formatted at 2026-09-26 16:36:29 on dist-test-slave-2zvq"
server_key: "3a2873584763b8b9581e3997c6ade545"
server_key_iv: "67501cc9d4e751a228ce3904482057f6"
server_key_version: "encryptionkey@0"
I20260926 16:36:29.500516 32455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "ae09c4534b6f47ebaa871e6ec8f6a4ca"
format_stamp: "Formatted at 2026-09-26 16:36:29 on dist-test-slave-2zvq"
server_key: "3a2873584763b8b9581e3997c6ade545"
server_key_iv: "67501cc9d4e751a228ce3904482057f6"
server_key_version: "encryptionkey@0"
I20260926 16:36:29.504146 32455 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 16:36:29.506559 32470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:29.507732 32455 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260926 16:36:29.507891 32455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "ae09c4534b6f47ebaa871e6ec8f6a4ca"
format_stamp: "Formatted at 2026-09-26 16:36:29 on dist-test-slave-2zvq"
server_key: "3a2873584763b8b9581e3997c6ade545"
server_key_iv: "67501cc9d4e751a228ce3904482057f6"
server_key_version: "encryptionkey@0"
I20260926 16:36:29.508002 32455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:29.512444 32455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:29.515232 32455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:29.515419 32455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:29.522051 32455 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36235
I20260926 16:36:29.522061 32522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36235 every 8 connection(s)
I20260926 16:36:29.522965 32455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:36:29.525528 32523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:29.528712 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 32455
I20260926 16:36:29.528848 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:36:29.529114 11299 external_mini_cluster.cc:1428] Setting key 100259726d499293723413bdec87cf6f
I20260926 16:36:29.530941 32523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae09c4534b6f47ebaa871e6ec8f6a4ca: Bootstrap starting.
I20260926 16:36:29.533327 32523 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae09c4534b6f47ebaa871e6ec8f6a4ca: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:29.534049 32523 log.cc:824] T 00000000000000000000000000000000 P ae09c4534b6f47ebaa871e6ec8f6a4ca: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:29.534209 32526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:29.530413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.171:52289 (local address 127.0.0.1:36235)
0926 16:36:29.530812 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:29.530832 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:29.530865 (+ 33us) server_negotiation.cc:407] Connection header received
0926 16:36:29.531559 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:29.531577 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:29.531853 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:29.532118 (+ 265us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:29.532952 (+ 834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:29.532957 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:29.532967 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:36:29.532978 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:29.533029 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:36:29.533048 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:29.533137 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:29.533288 (+ 151us) server_negotiation.cc:299] Negotiation successful
0926 16:36:29.533470 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":130,"threads_started":1}
I20260926 16:36:29.536099 32523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae09c4534b6f47ebaa871e6ec8f6a4ca: No bootstrap required, opened a new log
I20260926 16:36:29.537122 11299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:36:29.539955 32523 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae09c4534b6f47ebaa871e6ec8f6a4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae09c4534b6f47ebaa871e6ec8f6a4ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36235 } }
I20260926 16:36:29.540180 32523 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae09c4534b6f47ebaa871e6ec8f6a4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:36:29.540258 32523 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae09c4534b6f47ebaa871e6ec8f6a4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae09c4534b6f47ebaa871e6ec8f6a4ca, State: Initialized, Role: FOLLOWER
I20260926 16:36:29.540721 32523 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae09c4534b6f47ebaa871e6ec8f6a4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae09c4534b6f47ebaa871e6ec8f6a4ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36235 } }
I20260926 16:36:29.540872 32523 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae09c4534b6f47ebaa871e6ec8f6a4ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:36:29.540977 32523 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae09c4534b6f47ebaa871e6ec8f6a4ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:36:29.540999 32526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:29.539192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.171:33199 (local address 127.0.0.1:36235)
0926 16:36:29.539312 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:29.539315 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:29.539572 (+ 257us) server_negotiation.cc:407] Connection header received
0926 16:36:29.539723 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:29.539726 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:29.539764 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:29.539872 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:29.540625 (+ 753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:29.540628 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:29.540631 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:36:29.540642 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:29.540666 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:36:29.540677 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:29.540774 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:29.540846 (+ 72us) server_negotiation.cc:299] Negotiation successful
0926 16:36:29.540885 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 16:36:29.541098 32523 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae09c4534b6f47ebaa871e6ec8f6a4ca [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:29.542032 32523 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae09c4534b6f47ebaa871e6ec8f6a4ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae09c4534b6f47ebaa871e6ec8f6a4ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36235 } }
I20260926 16:36:29.542418 32523 leader_election.cc:302] T 00000000000000000000000000000000 P ae09c4534b6f47ebaa871e6ec8f6a4ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae09c4534b6f47ebaa871e6ec8f6a4ca; no voters:
I20260926 16:36:29.542693 32523 leader_election.cc:288] T 00000000000000000000000000000000 P ae09c4534b6f47ebaa871e6ec8f6a4ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:36:29.542796 32533 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae09c4534b6f47ebaa871e6ec8f6a4ca [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:36:29.543071 32533 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae09c4534b6f47ebaa871e6ec8f6a4ca [term 1 LEADER]: Becoming Leader. State: Replica: ae09c4534b6f47ebaa871e6ec8f6a4ca, State: Running, Role: LEADER
I20260926 16:36:29.543404 32533 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae09c4534b6f47ebaa871e6ec8f6a4ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae09c4534b6f47ebaa871e6ec8f6a4ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36235 } }
I20260926 16:36:29.543836 32523 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae09c4534b6f47ebaa871e6ec8f6a4ca [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:36:29.544998 32534 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae09c4534b6f47ebaa871e6ec8f6a4ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae09c4534b6f47ebaa871e6ec8f6a4ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae09c4534b6f47ebaa871e6ec8f6a4ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36235 } } }
I20260926 16:36:29.545132 32534 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae09c4534b6f47ebaa871e6ec8f6a4ca [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:29.545454 32535 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae09c4534b6f47ebaa871e6ec8f6a4ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae09c4534b6f47ebaa871e6ec8f6a4ca. Latest consensus state: current_term: 1 leader_uuid: "ae09c4534b6f47ebaa871e6ec8f6a4ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae09c4534b6f47ebaa871e6ec8f6a4ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36235 } } }
I20260926 16:36:29.545619 32535 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae09c4534b6f47ebaa871e6ec8f6a4ca [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:29.546231 32542 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:36:29.548760 32542 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:36:29.553779 32542 catalog_manager.cc:1347] Generated new cluster ID: 4b220d4f53ec47468039fc8be957dc15
I20260926 16:36:29.553851 32542 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:36:29.578992 32542 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:36:29.579350 32542 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:36:29.592119 32542 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae09c4534b6f47ebaa871e6ec8f6a4ca: Generated new TSK 0
I20260926 16:36:29.592893 32542 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:36:29.596880 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 32455
2026-09-26T16:36:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1385 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:36:29 dist-test-slave-2zvq krb5kdc[32554](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:36:29 dist-test-slave-2zvq krb5kdc[32554](info): set up 2 sockets
Sep 26 16:36:29 dist-test-slave-2zvq krb5kdc[32554](info): commencing operation
krb5kdc: starting...
W20260926 16:36:30.636483 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:36:30 dist-test-slave-2zvq krb5kdc[32554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:36:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:36:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 16:36:30.755299 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45263
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45263
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:36:30.863353 32567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:30.863641 32567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:30.863715 32567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:30.867686 32567 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:36:30.867761 32567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:30.867799 32567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:30.867856 32567 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:36:30.867895 32567 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:36:30.873029 32567 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45263
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:30.874020 32567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:30.875115 32567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:30.883423 32572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:30.883473 32575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:30.883431 32573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:30.883639 32567 server_base.cc:1034] running on GCE node
I20260926 16:36:30.884197 32567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:30.885262 32567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:30.886420 32567 hybrid_clock.cc:648] HybridClock initialized: now 1790440590886403 us; error 29 us; skew 500 ppm
Sep 26 16:36:30 dist-test-slave-2zvq krb5kdc[32554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440590, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:36:30.889034 32567 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 16:36:30.889858 32567 webserver.cc:460] Webserver started at http://127.0.0.1:35795/ using document root <none> and password file <none>
I20260926 16:36:30.890450 32567 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:30.890538 32567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:30.890748 32567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:30.892658 32567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "765299f5acfd49ed9218c835c742c90c"
format_stamp: "Formatted at 2026-09-26 16:36:30 on dist-test-slave-2zvq"
server_key: "2e0fe3ac0f3594aedb8b47a26064cf7c"
server_key_iv: "8f3197bc958b482c492038ecf5209fbc"
server_key_version: "encryptionkey@0"
I20260926 16:36:30.893209 32567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "765299f5acfd49ed9218c835c742c90c"
format_stamp: "Formatted at 2026-09-26 16:36:30 on dist-test-slave-2zvq"
server_key: "2e0fe3ac0f3594aedb8b47a26064cf7c"
server_key_iv: "8f3197bc958b482c492038ecf5209fbc"
server_key_version: "encryptionkey@0"
I20260926 16:36:30.896803 32567 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260926 16:36:30.899194 32582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:30.900403 32567 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260926 16:36:30.900544 32567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "765299f5acfd49ed9218c835c742c90c"
format_stamp: "Formatted at 2026-09-26 16:36:30 on dist-test-slave-2zvq"
server_key: "2e0fe3ac0f3594aedb8b47a26064cf7c"
server_key_iv: "8f3197bc958b482c492038ecf5209fbc"
server_key_version: "encryptionkey@0"
I20260926 16:36:30.900648 32567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:30.926187 32567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:30.929176 32567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:30.929373 32567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:30.936152 32567 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45263
I20260926 16:36:30.936175 32634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45263 every 8 connection(s)
I20260926 16:36:30.937072 32567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:36:30.939910 32635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:30.940930 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 32567
I20260926 16:36:30.941076 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:36:30.941457 11299 external_mini_cluster.cc:1428] Setting key 0425c986251fbe84f1a16d884a4ee556
I20260926 16:36:30.947268 32635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 765299f5acfd49ed9218c835c742c90c: Bootstrap starting.
I20260926 16:36:30.949394 32635 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 765299f5acfd49ed9218c835c742c90c: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:30.950217 32635 log.cc:824] T 00000000000000000000000000000000 P 765299f5acfd49ed9218c835c742c90c: Log is configured to *not* fsync() on all Append() calls
Sep 26 16:36:30 dist-test-slave-2zvq krb5kdc[32554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 16:36:30.952190 32635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 765299f5acfd49ed9218c835c742c90c: No bootstrap required, opened a new log
I20260926 16:36:30.955776 32635 raft_consensus.cc:348] T 00000000000000000000000000000000 P 765299f5acfd49ed9218c835c742c90c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "765299f5acfd49ed9218c835c742c90c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45263 } }
I20260926 16:36:30.956054 32635 raft_consensus.cc:374] T 00000000000000000000000000000000 P 765299f5acfd49ed9218c835c742c90c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:36:30.956153 32635 raft_consensus.cc:729] T 00000000000000000000000000000000 P 765299f5acfd49ed9218c835c742c90c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 765299f5acfd49ed9218c835c742c90c, State: Initialized, Role: FOLLOWER
I20260926 16:36:30.956645 32635 consensus_queue.cc:260] T 00000000000000000000000000000000 P 765299f5acfd49ed9218c835c742c90c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "765299f5acfd49ed9218c835c742c90c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45263 } }
I20260926 16:36:30.956815 32635 raft_consensus.cc:388] T 00000000000000000000000000000000 P 765299f5acfd49ed9218c835c742c90c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:36:30.956902 32635 raft_consensus.cc:482] T 00000000000000000000000000000000 P 765299f5acfd49ed9218c835c742c90c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:36:30.957000 32635 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 765299f5acfd49ed9218c835c742c90c [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:30.957312 32638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:30.943064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.171:35243 (local address 127.0.0.1:45263)
0926 16:36:30.943538 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:30.943562 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:30.943624 (+ 62us) server_negotiation.cc:407] Connection header received
0926 16:36:30.944640 (+ 1016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:30.944672 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:30.945192 (+ 520us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:30.945601 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:30.946538 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:30.947653 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:30.948600 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:30.949109 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:30.949301 (+ 192us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:30.952206 (+ 2905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:30.952211 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:30.952224 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:30.952258 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:30.954019 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:30.954641 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:30.954646 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:30.954651 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:30.954766 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:30.955148 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:30.955151 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:30.955152 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:30.955540 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:30.955802 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:30.956095 (+ 293us) server_negotiation.cc:299] Negotiation successful
0926 16:36:30.956365 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":127,"threads_started":1}
I20260926 16:36:30.957963 32635 raft_consensus.cc:504] T 00000000000000000000000000000000 P 765299f5acfd49ed9218c835c742c90c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "765299f5acfd49ed9218c835c742c90c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45263 } }
I20260926 16:36:30.958354 32635 leader_election.cc:302] T 00000000000000000000000000000000 P 765299f5acfd49ed9218c835c742c90c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 765299f5acfd49ed9218c835c742c90c; no voters:
I20260926 16:36:30.958693 32635 leader_election.cc:288] T 00000000000000000000000000000000 P 765299f5acfd49ed9218c835c742c90c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:36:30.958801 32640 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 765299f5acfd49ed9218c835c742c90c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:36:30.959005 32640 raft_consensus.cc:686] T 00000000000000000000000000000000 P 765299f5acfd49ed9218c835c742c90c [term 1 LEADER]: Becoming Leader. State: Replica: 765299f5acfd49ed9218c835c742c90c, State: Running, Role: LEADER
I20260926 16:36:30.959398 32640 consensus_queue.cc:237] T 00000000000000000000000000000000 P 765299f5acfd49ed9218c835c742c90c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "765299f5acfd49ed9218c835c742c90c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45263 } }
I20260926 16:36:30.959995 32635 sys_catalog.cc:564] T 00000000000000000000000000000000 P 765299f5acfd49ed9218c835c742c90c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:36:30.960808 11299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:36:30.961087 32642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 765299f5acfd49ed9218c835c742c90c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 765299f5acfd49ed9218c835c742c90c. Latest consensus state: current_term: 1 leader_uuid: "765299f5acfd49ed9218c835c742c90c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "765299f5acfd49ed9218c835c742c90c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45263 } } }
I20260926 16:36:30.961197 32642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 765299f5acfd49ed9218c835c742c90c [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:30.961576 32641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 765299f5acfd49ed9218c835c742c90c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "765299f5acfd49ed9218c835c742c90c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "765299f5acfd49ed9218c835c742c90c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45263 } } }
I20260926 16:36:30.961655 32641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 765299f5acfd49ed9218c835c742c90c [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:30.961934 32649 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:36:30.964859 32649 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:36:30.970260 32649 catalog_manager.cc:1347] Generated new cluster ID: b96eb69921d14eea8f070bfeb85a45cc
I20260926 16:36:30.970381 32649 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:36:30.975418 32638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:30.963273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.171:55481 (local address 127.0.0.1:45263)
0926 16:36:30.963376 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:30.963383 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:30.966560 (+ 3177us) server_negotiation.cc:407] Connection header received
0926 16:36:30.966747 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:30.966753 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:30.966805 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:30.966894 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:30.967582 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:30.968575 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:30.969768 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:30.970113 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:30.970220 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:30.971907 (+ 1687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:30.971911 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:30.971913 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:30.971938 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:30.973497 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:30.974003 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:30.974008 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:30.974010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:30.974096 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:30.974487 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:30.974490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:30.974492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:30.974671 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:30.974864 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:30.975137 (+ 273us) server_negotiation.cc:299] Negotiation successful
0926 16:36:30.975231 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 16:36:30.990757 32649 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:36:30.991665 32649 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:36:30.998236 32649 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 765299f5acfd49ed9218c835c742c90c: Generated new TSK 0
I20260926 16:36:30.998986 32649 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:36:31.005630 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 32567
W20260926 16:36:31.010252 32552 connection.cc:535] client connection to 127.0.0.1:45263 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45263): Connection reset by peer (error 104)
2026-09-26T16:36:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1408 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:36:31 dist-test-slave-2zvq krb5kdc[32666](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:36:31 dist-test-slave-2zvq krb5kdc[32666](info): set up 2 sockets
Sep 26 16:36:31 dist-test-slave-2zvq krb5kdc[32666](info): commencing operation
krb5kdc: starting...
W20260926 16:36:32.052565 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:36:32 dist-test-slave-2zvq krb5kdc[32666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:36:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:36:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 16:36:32.171554 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39647
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39647
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:36:32.278821 32679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:32.279096 32679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:32.279138 32679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:32.283067 32679 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:36:32.283116 32679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:32.283135 32679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:32.283151 32679 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:36:32.283165 32679 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:36:32.288165 32679 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39647
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:32.289069 32679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:32.289974 32679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:32.298252 32684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:32.298225 32687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:32.298308 32679 server_base.cc:1034] running on GCE node
W20260926 16:36:32.298235 32685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:32.299072 32679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:32.300168 32679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:32.301355 32679 hybrid_clock.cc:648] HybridClock initialized: now 1790440592301342 us; error 28 us; skew 500 ppm
Sep 26 16:36:32 dist-test-slave-2zvq krb5kdc[32666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440592, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:36:32.304024 32679 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 16:36:32.304854 32679 webserver.cc:460] Webserver started at http://127.0.0.1:44063/ using document root <none> and password file <none>
I20260926 16:36:32.305413 32679 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:32.305498 32679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:32.305712 32679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:32.307706 32679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "41a915aa604c41f3a85a365afa0d69f1"
format_stamp: "Formatted at 2026-09-26 16:36:32 on dist-test-slave-2zvq"
server_key: "ab59093e2b1954cb104bc58998c7bb1a"
server_key_iv: "74df18f46ad1fa7dfbcf6e7f48d021ed"
server_key_version: "encryptionkey@0"
I20260926 16:36:32.308257 32679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "41a915aa604c41f3a85a365afa0d69f1"
format_stamp: "Formatted at 2026-09-26 16:36:32 on dist-test-slave-2zvq"
server_key: "ab59093e2b1954cb104bc58998c7bb1a"
server_key_iv: "74df18f46ad1fa7dfbcf6e7f48d021ed"
server_key_version: "encryptionkey@0"
I20260926 16:36:32.311766 32679 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 16:36:32.314095 32694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:32.315253 32679 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:36:32.315397 32679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "41a915aa604c41f3a85a365afa0d69f1"
format_stamp: "Formatted at 2026-09-26 16:36:32 on dist-test-slave-2zvq"
server_key: "ab59093e2b1954cb104bc58998c7bb1a"
server_key_iv: "74df18f46ad1fa7dfbcf6e7f48d021ed"
server_key_version: "encryptionkey@0"
I20260926 16:36:32.315501 32679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:32.335976 32679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:32.338624 32679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:32.338810 32679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:32.345450 32679 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39647
I20260926 16:36:32.345482 32746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39647 every 8 connection(s)
I20260926 16:36:32.346342 32679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:36:32.347110 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 32679
I20260926 16:36:32.347242 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:36:32.347536 11299 external_mini_cluster.cc:1428] Setting key 8173231401337ee13a61efa3b2ed9130
I20260926 16:36:32.349434 32747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:32.354565 32747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41a915aa604c41f3a85a365afa0d69f1: Bootstrap starting.
I20260926 16:36:32.355954 32750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:32.348820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.171:39769 (local address 127.0.0.1:39647)
0926 16:36:32.349238 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:32.349279 (+ 41us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:32.349313 (+ 34us) server_negotiation.cc:407] Connection header received
0926 16:36:32.349960 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:32.349977 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:32.350356 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:32.350682 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:32.351053 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:32.352008 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:32.352883 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:32.353335 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:32.353510 (+ 175us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:32.354388 (+ 878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:32.354392 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:32.354403 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:36:32.354799 (+ 396us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:32.354850 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:36:32.354872 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:32.355006 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:32.355180 (+ 174us) server_negotiation.cc:299] Negotiation successful
0926 16:36:32.355311 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":134,"threads_started":1}
I20260926 16:36:32.356693 32747 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41a915aa604c41f3a85a365afa0d69f1: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:32.357375 32747 log.cc:824] T 00000000000000000000000000000000 P 41a915aa604c41f3a85a365afa0d69f1: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:32.358827 11299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:36:32.359323 32747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41a915aa604c41f3a85a365afa0d69f1: No bootstrap required, opened a new log
I20260926 16:36:32.362893 32747 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41a915aa604c41f3a85a365afa0d69f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41a915aa604c41f3a85a365afa0d69f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39647 } }
I20260926 16:36:32.363109 32747 raft_consensus.cc:374] T 00000000000000000000000000000000 P 41a915aa604c41f3a85a365afa0d69f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:36:32.363216 32747 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41a915aa604c41f3a85a365afa0d69f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41a915aa604c41f3a85a365afa0d69f1, State: Initialized, Role: FOLLOWER
I20260926 16:36:32.363663 32747 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41a915aa604c41f3a85a365afa0d69f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41a915aa604c41f3a85a365afa0d69f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39647 } }
I20260926 16:36:32.363847 32747 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41a915aa604c41f3a85a365afa0d69f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:36:32.363920 32747 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41a915aa604c41f3a85a365afa0d69f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:36:32.364022 32747 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41a915aa604c41f3a85a365afa0d69f1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:32.364444 32750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:32.360700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.171:51205 (local address 127.0.0.1:39647)
0926 16:36:32.360804 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:32.360809 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:32.360945 (+ 136us) server_negotiation.cc:407] Connection header received
0926 16:36:32.361080 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:32.361084 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:32.361122 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:32.361209 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:32.361454 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:32.361945 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:32.362808 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:32.363054 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:32.363155 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:32.364026 (+ 871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:32.364032 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:32.364037 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:36:32.364051 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:32.364077 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:36:32.364092 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:32.364193 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:32.364294 (+ 101us) server_negotiation.cc:299] Negotiation successful
0926 16:36:32.364327 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 16:36:32.365038 32747 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41a915aa604c41f3a85a365afa0d69f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41a915aa604c41f3a85a365afa0d69f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39647 } }
I20260926 16:36:32.365355 32747 leader_election.cc:302] T 00000000000000000000000000000000 P 41a915aa604c41f3a85a365afa0d69f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41a915aa604c41f3a85a365afa0d69f1; no voters:
I20260926 16:36:32.365626 32747 leader_election.cc:288] T 00000000000000000000000000000000 P 41a915aa604c41f3a85a365afa0d69f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:36:32.365713 32757 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41a915aa604c41f3a85a365afa0d69f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:36:32.365933 32757 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41a915aa604c41f3a85a365afa0d69f1 [term 1 LEADER]: Becoming Leader. State: Replica: 41a915aa604c41f3a85a365afa0d69f1, State: Running, Role: LEADER
I20260926 16:36:32.366237 32757 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41a915aa604c41f3a85a365afa0d69f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41a915aa604c41f3a85a365afa0d69f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39647 } }
I20260926 16:36:32.366762 32747 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41a915aa604c41f3a85a365afa0d69f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:36:32.367756 32758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41a915aa604c41f3a85a365afa0d69f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "41a915aa604c41f3a85a365afa0d69f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41a915aa604c41f3a85a365afa0d69f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39647 } } }
I20260926 16:36:32.367887 32758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41a915aa604c41f3a85a365afa0d69f1 [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:32.368194 32759 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41a915aa604c41f3a85a365afa0d69f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41a915aa604c41f3a85a365afa0d69f1. Latest consensus state: current_term: 1 leader_uuid: "41a915aa604c41f3a85a365afa0d69f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41a915aa604c41f3a85a365afa0d69f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39647 } } }
I20260926 16:36:32.368284 32766 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:36:32.368352 32759 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41a915aa604c41f3a85a365afa0d69f1 [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:32.371523 32766 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:36:32.376832 32766 catalog_manager.cc:1347] Generated new cluster ID: 80693e95239a424788fdd207afe99f0d
I20260926 16:36:32.376904 32766 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:36:32.384792 32766 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:36:32.385676 32766 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:36:32.392484 32766 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 41a915aa604c41f3a85a365afa0d69f1: Generated new TSK 0
I20260926 16:36:32.393137 32766 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:36:32.397830 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 32679
W20260926 16:36:32.403231 32664 connection.cc:535] client connection to 127.0.0.1:39647 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39647): Connection reset by peer (error 104)
2026-09-26T16:36:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1393 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:36:32 dist-test-slave-2zvq krb5kdc[310](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:36:32 dist-test-slave-2zvq krb5kdc[310](info): set up 2 sockets
Sep 26 16:36:32 dist-test-slave-2zvq krb5kdc[310](info): commencing operation
krb5kdc: starting...
W20260926 16:36:33.436152 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:36:33 dist-test-slave-2zvq krb5kdc[310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440593, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:36:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:36:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 16:36:33.553045 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42001
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42001
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:36:33.661422 323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:33.661711 323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:33.661787 323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:33.665735 323 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:36:33.665818 323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:33.665855 323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:33.665906 323 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:36:33.665946 323 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:36:33.671113 323 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42001
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440324823152-11299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:33.672179 323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:33.673164 323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:33.681645 328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:33.681640 331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:33.681640 329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:33.681711 323 server_base.cc:1034] running on GCE node
I20260926 16:36:33.682507 323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:33.683550 323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:33.684736 323 hybrid_clock.cc:648] HybridClock initialized: now 1790440593684708 us; error 48 us; skew 500 ppm
Sep 26 16:36:33 dist-test-slave-2zvq krb5kdc[310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440593, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:36:33.693892 323 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 16:36:33.694850 323 webserver.cc:460] Webserver started at http://127.0.0.1:44139/ using document root <none> and password file <none>
I20260926 16:36:33.695392 323 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:33.695451 323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:33.695612 323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:33.697564 323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "0a678183b47e499988bbdd2e2652a69f"
format_stamp: "Formatted at 2026-09-26 16:36:33 on dist-test-slave-2zvq"
server_key: "afbe1b321f42bc385f6cefcc84a532da"
server_key_iv: "33277329a3a324e57feb45e131151f99"
server_key_version: "encryptionkey@0"
I20260926 16:36:33.698050 323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "0a678183b47e499988bbdd2e2652a69f"
format_stamp: "Formatted at 2026-09-26 16:36:33 on dist-test-slave-2zvq"
server_key: "afbe1b321f42bc385f6cefcc84a532da"
server_key_iv: "33277329a3a324e57feb45e131151f99"
server_key_version: "encryptionkey@0"
I20260926 16:36:33.701617 323 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260926 16:36:33.703927 338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:33.705128 323 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 16:36:33.705250 323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "0a678183b47e499988bbdd2e2652a69f"
format_stamp: "Formatted at 2026-09-26 16:36:33 on dist-test-slave-2zvq"
server_key: "afbe1b321f42bc385f6cefcc84a532da"
server_key_iv: "33277329a3a324e57feb45e131151f99"
server_key_version: "encryptionkey@0"
I20260926 16:36:33.705375 323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:33.710208 323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:33.713507 323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:33.713701 323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:33.720912 323 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42001
I20260926 16:36:33.720968 390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42001 every 8 connection(s)
I20260926 16:36:33.721802 323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:36:33.724457 391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:33.728863 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 323
I20260926 16:36:33.729019 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:36:33.729343 11299 external_mini_cluster.cc:1428] Setting key 85943118356896127546c5e6ae8f18f0
I20260926 16:36:33.729691 391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a678183b47e499988bbdd2e2652a69f: Bootstrap starting.
I20260926 16:36:33.732059 391 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a678183b47e499988bbdd2e2652a69f: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:33.732905 391 log.cc:824] T 00000000000000000000000000000000 P 0a678183b47e499988bbdd2e2652a69f: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:33.734648 394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:33.730713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.171:55983 (local address 127.0.0.1:42001)
0926 16:36:33.731140 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:33.731154 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:33.731191 (+ 37us) server_negotiation.cc:407] Connection header received
0926 16:36:33.731848 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:33.731865 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:33.732180 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:33.732493 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:33.733356 (+ 863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:33.733362 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:33.733379 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:36:33.733393 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:33.733464 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:36:33.733500 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:33.733626 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:33.733732 (+ 106us) server_negotiation.cc:299] Negotiation successful
0926 16:36:33.733881 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":135,"threads_started":1}
I20260926 16:36:33.735100 391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a678183b47e499988bbdd2e2652a69f: No bootstrap required, opened a new log
I20260926 16:36:33.737368 11299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:36:33.738973 391 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a678183b47e499988bbdd2e2652a69f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a678183b47e499988bbdd2e2652a69f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42001 } }
I20260926 16:36:33.739228 391 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a678183b47e499988bbdd2e2652a69f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:36:33.739300 391 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a678183b47e499988bbdd2e2652a69f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a678183b47e499988bbdd2e2652a69f, State: Initialized, Role: FOLLOWER
I20260926 16:36:33.739818 391 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a678183b47e499988bbdd2e2652a69f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a678183b47e499988bbdd2e2652a69f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42001 } }
I20260926 16:36:33.740003 391 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a678183b47e499988bbdd2e2652a69f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:36:33.740074 391 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a678183b47e499988bbdd2e2652a69f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:36:33.740170 391 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a678183b47e499988bbdd2e2652a69f [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:33.741048 391 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a678183b47e499988bbdd2e2652a69f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a678183b47e499988bbdd2e2652a69f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42001 } }
I20260926 16:36:33.741258 394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:33.739420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.171:49193 (local address 127.0.0.1:42001)
0926 16:36:33.739556 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:33.739560 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:33.739765 (+ 205us) server_negotiation.cc:407] Connection header received
0926 16:36:33.739877 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:33.739879 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:33.739911 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:33.740029 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:33.740804 (+ 775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:33.740808 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:33.740811 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:36:33.740824 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:33.740858 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:36:33.740869 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:33.740966 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:33.741045 (+ 79us) server_negotiation.cc:299] Negotiation successful
0926 16:36:33.741091 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 16:36:33.741426 391 leader_election.cc:302] T 00000000000000000000000000000000 P 0a678183b47e499988bbdd2e2652a69f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a678183b47e499988bbdd2e2652a69f; no voters:
I20260926 16:36:33.741750 391 leader_election.cc:288] T 00000000000000000000000000000000 P 0a678183b47e499988bbdd2e2652a69f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:36:33.741818 401 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a678183b47e499988bbdd2e2652a69f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:36:33.742017 401 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a678183b47e499988bbdd2e2652a69f [term 1 LEADER]: Becoming Leader. State: Replica: 0a678183b47e499988bbdd2e2652a69f, State: Running, Role: LEADER
I20260926 16:36:33.742360 401 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a678183b47e499988bbdd2e2652a69f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a678183b47e499988bbdd2e2652a69f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42001 } }
I20260926 16:36:33.743062 391 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a678183b47e499988bbdd2e2652a69f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:36:33.744233 403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a678183b47e499988bbdd2e2652a69f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a678183b47e499988bbdd2e2652a69f. Latest consensus state: current_term: 1 leader_uuid: "0a678183b47e499988bbdd2e2652a69f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a678183b47e499988bbdd2e2652a69f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42001 } } }
I20260926 16:36:33.744266 402 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a678183b47e499988bbdd2e2652a69f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a678183b47e499988bbdd2e2652a69f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a678183b47e499988bbdd2e2652a69f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42001 } } }
I20260926 16:36:33.744445 403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a678183b47e499988bbdd2e2652a69f [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:33.744464 402 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a678183b47e499988bbdd2e2652a69f [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:33.744879 411 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:36:33.748044 411 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:36:33.752946 411 catalog_manager.cc:1347] Generated new cluster ID: cb8704b1b2134615a3347a90e21cace0
I20260926 16:36:33.753006 411 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:36:33.780011 411 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:36:33.780388 411 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:36:33.785388 411 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a678183b47e499988bbdd2e2652a69f: Generated new TSK 0
I20260926 16:36:33.786037 411 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:36:33.796658 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 323
2026-09-26T16:36:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1398 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:36:33 dist-test-slave-2zvq krb5kdc[423](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:36:33 dist-test-slave-2zvq krb5kdc[423](info): set up 2 sockets
Sep 26 16:36:33 dist-test-slave-2zvq krb5kdc[423](info): commencing operation
krb5kdc: starting...
W20260926 16:36:34.856176 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:36:34 dist-test-slave-2zvq krb5kdc[423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:36:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:36:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
I20260926 16:36:34.977114 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:32967
--webserver_interface=127.11.8.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:44625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:32967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:36:35.084998 436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:35.085291 436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:35.085366 436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:35.089414 436 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:36:35.089507 436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:35.089545 436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:35.089599 436 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:36:35.089639 436 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:36:35.095007 436 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:44625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:32967
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:32967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.436
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:35.096082 436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:35.097103 436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:35.105561 442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:35.105544 444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:35.105835 436 server_base.cc:1034] running on GCE node
W20260926 16:36:35.105535 441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:35.106534 436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:35.107605 436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:35.108798 436 hybrid_clock.cc:648] HybridClock initialized: now 1790440595108781 us; error 31 us; skew 500 ppm
Sep 26 16:36:35 dist-test-slave-2zvq krb5kdc[423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440595, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:36:35.111809 436 init.cc:377] Logged in from keytab as kudu/127.11.8.254@KRBTEST.COM (short username kudu)
I20260926 16:36:35.112744 436 webserver.cc:460] Webserver started at http://127.11.8.254:43431/ using document root <none> and password file <none>
I20260926 16:36:35.113345 436 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:35.113442 436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:35.113683 436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:35.115901 436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "03f7a719c932478f99d2261f6f08a055"
format_stamp: "Formatted at 2026-09-26 16:36:35 on dist-test-slave-2zvq"
server_key: "fa6b0c2e92cdd1aeb3506117e5cb4b56"
server_key_iv: "76e984cd1fd883206403308226d332ff"
server_key_version: "encryptionkey@0"
I20260926 16:36:35.116510 436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "03f7a719c932478f99d2261f6f08a055"
format_stamp: "Formatted at 2026-09-26 16:36:35 on dist-test-slave-2zvq"
server_key: "fa6b0c2e92cdd1aeb3506117e5cb4b56"
server_key_iv: "76e984cd1fd883206403308226d332ff"
server_key_version: "encryptionkey@0"
I20260926 16:36:35.120478 436 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 16:36:35.123056 451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:35.124296 436 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:36:35.124487 436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "03f7a719c932478f99d2261f6f08a055"
format_stamp: "Formatted at 2026-09-26 16:36:35 on dist-test-slave-2zvq"
server_key: "fa6b0c2e92cdd1aeb3506117e5cb4b56"
server_key_iv: "76e984cd1fd883206403308226d332ff"
server_key_version: "encryptionkey@0"
I20260926 16:36:35.124598 436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:35.152166 436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:35.155212 436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:35.155416 436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:35.162420 436 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:32967
I20260926 16:36:35.162456 503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:32967 every 8 connection(s)
I20260926 16:36:35.163570 436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:36:35.166245 504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:35.171648 504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03f7a719c932478f99d2261f6f08a055: Bootstrap starting.
I20260926 16:36:35.172972 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 436
I20260926 16:36:35.173115 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:36:35.173412 11299 external_mini_cluster.cc:1428] Setting key d0412604b8e7fb84997a4b3dcfe1617c
I20260926 16:36:35.174172 504 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03f7a719c932478f99d2261f6f08a055: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:35.175055 504 log.cc:824] T 00000000000000000000000000000000 P 03f7a719c932478f99d2261f6f08a055: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:35.177281 504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03f7a719c932478f99d2261f6f08a055: No bootstrap required, opened a new log
I20260926 16:36:35.181614 504 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03f7a719c932478f99d2261f6f08a055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03f7a719c932478f99d2261f6f08a055" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32967 } }
I20260926 16:36:35.181916 504 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03f7a719c932478f99d2261f6f08a055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:36:35.181980 504 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03f7a719c932478f99d2261f6f08a055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03f7a719c932478f99d2261f6f08a055, State: Initialized, Role: FOLLOWER
Sep 26 16:36:35 dist-test-slave-2zvq krb5kdc[423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:36:35.182457 504 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03f7a719c932478f99d2261f6f08a055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03f7a719c932478f99d2261f6f08a055" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32967 } }
I20260926 16:36:35.182595 504 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03f7a719c932478f99d2261f6f08a055 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:36:35.182665 504 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03f7a719c932478f99d2261f6f08a055 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:36:35.182788 504 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03f7a719c932478f99d2261f6f08a055 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:35.183658 504 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03f7a719c932478f99d2261f6f08a055 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03f7a719c932478f99d2261f6f08a055" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32967 } }
I20260926 16:36:35.184041 504 leader_election.cc:302] T 00000000000000000000000000000000 P 03f7a719c932478f99d2261f6f08a055 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03f7a719c932478f99d2261f6f08a055; no voters:
I20260926 16:36:35.184381 504 leader_election.cc:288] T 00000000000000000000000000000000 P 03f7a719c932478f99d2261f6f08a055 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:36:35.184515 510 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03f7a719c932478f99d2261f6f08a055 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:36:35.184728 510 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03f7a719c932478f99d2261f6f08a055 [term 1 LEADER]: Becoming Leader. State: Replica: 03f7a719c932478f99d2261f6f08a055, State: Running, Role: LEADER
I20260926 16:36:35.185091 510 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03f7a719c932478f99d2261f6f08a055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03f7a719c932478f99d2261f6f08a055" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32967 } }
I20260926 16:36:35.185534 504 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03f7a719c932478f99d2261f6f08a055 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:36:35.186398 510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03f7a719c932478f99d2261f6f08a055 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03f7a719c932478f99d2261f6f08a055. Latest consensus state: current_term: 1 leader_uuid: "03f7a719c932478f99d2261f6f08a055" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03f7a719c932478f99d2261f6f08a055" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32967 } } }
I20260926 16:36:35.187114 510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03f7a719c932478f99d2261f6f08a055 [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:35.187005 511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03f7a719c932478f99d2261f6f08a055 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03f7a719c932478f99d2261f6f08a055" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03f7a719c932478f99d2261f6f08a055" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 32967 } } }
I20260926 16:36:35.187429 511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03f7a719c932478f99d2261f6f08a055 [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:35.189157 507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:35.174499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34149 (local address 127.11.8.254:32967)
0926 16:36:35.174892 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:35.174908 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:35.174942 (+ 34us) server_negotiation.cc:407] Connection header received
0926 16:36:35.175758 (+ 816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:35.175783 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:35.176145 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:35.176425 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:35.177333 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:35.178402 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:35.179589 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:35.180057 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:35.180175 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:35.183204 (+ 3029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:35.183209 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:35.183222 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:35.183249 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:35.185397 (+ 2148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:35.186005 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:35.186011 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:35.186016 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:35.186160 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:35.186627 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:35.186630 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:35.186632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:35.187200 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:35.188116 (+ 916us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:35.188247 (+ 131us) server_negotiation.cc:299] Negotiation successful
0926 16:36:35.188421 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":116,"threads_started":1}
W20260926 16:36:35.190769 525 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 03f7a719c932478f99d2261f6f08a055: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 16:36:35.190845 525 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 16:36:35.190908 522 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:36:35.191735 522 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:36:35.192099 11299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:36:35.197435 522 catalog_manager.cc:1347] Generated new cluster ID: 88d5f7d6e5844b139261ed0a0accdb48
I20260926 16:36:35.197494 522 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:36:35.202899 507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:35.194052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52297 (local address 127.11.8.254:32967)
0926 16:36:35.194144 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:35.194148 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:35.194426 (+ 278us) server_negotiation.cc:407] Connection header received
0926 16:36:35.194547 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:35.194550 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:35.194592 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:35.194673 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:35.195484 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:35.196412 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:35.197286 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:35.197635 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:35.197740 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:35.199472 (+ 1732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:35.199476 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:35.199479 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:35.199501 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:35.201115 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:35.201646 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:35.201649 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:35.201651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:35.201727 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:35.202050 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:35.202053 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:35.202054 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:35.202228 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:35.202393 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:35.202661 (+ 268us) server_negotiation.cc:299] Negotiation successful
0926 16:36:35.202750 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260926 16:36:35.213928 522 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:36:35.214967 522 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:36:35.220417 522 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03f7a719c932478f99d2261f6f08a055: Generated new TSK 0
I20260926 16:36:35.221094 522 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:36:35.229944 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 436
W20260926 16:36:35.235312 421 connection.cc:535] client connection to 127.11.8.254:32967 recv error: Network error: failed to read from TLS socket (remote: 127.11.8.254:32967): Connection reset by peer (error 104)
2026-09-26T16:36:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1433 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:36:35 dist-test-slave-2zvq krb5kdc[536](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:36:35 dist-test-slave-2zvq krb5kdc[536](info): set up 2 sockets
Sep 26 16:36:35 dist-test-slave-2zvq krb5kdc[536](info): commencing operation
krb5kdc: starting...
W20260926 16:36:36.276162 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:36:36 dist-test-slave-2zvq krb5kdc[536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:36:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:36:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
I20260926 16:36:36.392886 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:46777
--webserver_interface=127.11.8.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:33449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:46777
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:36:36.498013 549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:36.498256 549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:36.498391 549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:36.502247 549 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:36:36.502317 549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:36.502337 549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:36.502353 549 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:36:36.502374 549 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:36:36.507504 549 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:33449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:46777
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:46777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:36.508407 549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:36.509303 549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:36.517289 554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:36.517313 557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:36.517328 555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:36.517517 549 server_base.cc:1034] running on GCE node
I20260926 16:36:36.518042 549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:36.519095 549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:36.520256 549 hybrid_clock.cc:648] HybridClock initialized: now 1790440596520227 us; error 40 us; skew 500 ppm
Sep 26 16:36:36 dist-test-slave-2zvq krb5kdc[536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440596, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:36:36.522892 549 init.cc:377] Logged in from keytab as kudu/127.11.8.254@KRBTEST.COM (short username kudu)
I20260926 16:36:36.523749 549 webserver.cc:460] Webserver started at http://127.11.8.254:38563/ using document root <none> and password file <none>
I20260926 16:36:36.524292 549 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:36.524355 549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:36.524572 549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:36.526583 549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "84e1f723e7124013858da5971730ac1f"
format_stamp: "Formatted at 2026-09-26 16:36:36 on dist-test-slave-2zvq"
server_key: "555426854f784330cb30a14a1d85dce4"
server_key_iv: "11ab93f00d6379533d75ac78a579a8e5"
server_key_version: "encryptionkey@0"
I20260926 16:36:36.527140 549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "84e1f723e7124013858da5971730ac1f"
format_stamp: "Formatted at 2026-09-26 16:36:36 on dist-test-slave-2zvq"
server_key: "555426854f784330cb30a14a1d85dce4"
server_key_iv: "11ab93f00d6379533d75ac78a579a8e5"
server_key_version: "encryptionkey@0"
I20260926 16:36:36.530638 549 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260926 16:36:36.532862 564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:36.534107 549 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:36:36.534242 549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "84e1f723e7124013858da5971730ac1f"
format_stamp: "Formatted at 2026-09-26 16:36:36 on dist-test-slave-2zvq"
server_key: "555426854f784330cb30a14a1d85dce4"
server_key_iv: "11ab93f00d6379533d75ac78a579a8e5"
server_key_version: "encryptionkey@0"
I20260926 16:36:36.534395 549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:36.549690 549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:36.552524 549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:36.552718 549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:36.559580 549 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:46777
I20260926 16:36:36.559640 616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:46777 every 8 connection(s)
I20260926 16:36:36.560492 549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:36:36.563100 617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:36.568537 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 549
I20260926 16:36:36.568656 617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84e1f723e7124013858da5971730ac1f: Bootstrap starting.
I20260926 16:36:36.568691 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:36:36.569003 11299 external_mini_cluster.cc:1428] Setting key 7f7e0caf6552691ae11a8b6037aff6ce
I20260926 16:36:36.571239 617 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84e1f723e7124013858da5971730ac1f: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:36.571969 617 log.cc:824] T 00000000000000000000000000000000 P 84e1f723e7124013858da5971730ac1f: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:36.574139 617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84e1f723e7124013858da5971730ac1f: No bootstrap required, opened a new log
I20260926 16:36:36.578096 620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:36.570361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53079 (local address 127.11.8.254:46777)
0926 16:36:36.570774 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:36.570790 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:36.570825 (+ 35us) server_negotiation.cc:407] Connection header received
0926 16:36:36.571483 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:36.571500 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:36.571830 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:36.572148 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:36.572545 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:36.573590 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:36.574833 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:36.575323 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:36.575487 (+ 164us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:36.576388 (+ 901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:36.576393 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:36.576405 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:36:36.576778 (+ 373us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:36.576833 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:36:36.576856 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:36.576958 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:36.577179 (+ 221us) server_negotiation.cc:299] Negotiation successful
0926 16:36:36.577355 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":120,"threads_started":1}
I20260926 16:36:36.578228 617 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84e1f723e7124013858da5971730ac1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84e1f723e7124013858da5971730ac1f" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 46777 } }
I20260926 16:36:36.578533 617 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84e1f723e7124013858da5971730ac1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:36:36.578614 617 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84e1f723e7124013858da5971730ac1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84e1f723e7124013858da5971730ac1f, State: Initialized, Role: FOLLOWER
I20260926 16:36:36.579093 617 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84e1f723e7124013858da5971730ac1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84e1f723e7124013858da5971730ac1f" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 46777 } }
I20260926 16:36:36.579294 617 raft_consensus.cc:388] T 00000000000000000000000000000000 P 84e1f723e7124013858da5971730ac1f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:36:36.579473 617 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84e1f723e7124013858da5971730ac1f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:36:36.579578 617 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84e1f723e7124013858da5971730ac1f [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:36.580602 617 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84e1f723e7124013858da5971730ac1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84e1f723e7124013858da5971730ac1f" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 46777 } }
I20260926 16:36:36.580929 617 leader_election.cc:302] T 00000000000000000000000000000000 P 84e1f723e7124013858da5971730ac1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84e1f723e7124013858da5971730ac1f; no voters:
I20260926 16:36:36.581140 11299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:36:36.581254 617 leader_election.cc:288] T 00000000000000000000000000000000 P 84e1f723e7124013858da5971730ac1f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:36:36.581338 622 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84e1f723e7124013858da5971730ac1f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:36:36.581568 622 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84e1f723e7124013858da5971730ac1f [term 1 LEADER]: Becoming Leader. State: Replica: 84e1f723e7124013858da5971730ac1f, State: Running, Role: LEADER
I20260926 16:36:36.581868 622 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84e1f723e7124013858da5971730ac1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84e1f723e7124013858da5971730ac1f" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 46777 } }
I20260926 16:36:36.582684 617 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84e1f723e7124013858da5971730ac1f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:36:36.583460 624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84e1f723e7124013858da5971730ac1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84e1f723e7124013858da5971730ac1f. Latest consensus state: current_term: 1 leader_uuid: "84e1f723e7124013858da5971730ac1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84e1f723e7124013858da5971730ac1f" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 46777 } } }
I20260926 16:36:36.583595 624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84e1f723e7124013858da5971730ac1f [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:36.583981 636 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:36:36.583971 623 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84e1f723e7124013858da5971730ac1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "84e1f723e7124013858da5971730ac1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84e1f723e7124013858da5971730ac1f" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 46777 } } }
I20260926 16:36:36.584086 623 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84e1f723e7124013858da5971730ac1f [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:36.587291 636 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:36:36.588677 620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:36.583431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41071 (local address 127.11.8.254:46777)
0926 16:36:36.583556 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:36.583560 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:36.583861 (+ 301us) server_negotiation.cc:407] Connection header received
0926 16:36:36.584058 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:36.584061 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:36.584107 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:36.584161 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:36.584606 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:36.585530 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:36.586696 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:36.587060 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:36.587150 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:36.588223 (+ 1073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:36.588227 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:36.588230 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:36:36.588242 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:36.588273 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:36:36.588283 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:36.588383 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:36.588500 (+ 117us) server_negotiation.cc:299] Negotiation successful
0926 16:36:36.588532 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 16:36:36.593163 636 catalog_manager.cc:1347] Generated new cluster ID: b9e65035cc554254aaa572e0981a6271
I20260926 16:36:36.593223 636 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:36:36.601572 636 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:36:36.602519 636 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:36:36.611264 636 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 84e1f723e7124013858da5971730ac1f: Generated new TSK 0
I20260926 16:36:36.612466 636 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:36:36.620342 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 549
W20260926 16:36:36.625767 534 connection.cc:535] client connection to 127.11.8.254:46777 recv error: Network error: failed to read from TLS socket (remote: 127.11.8.254:46777): Connection reset by peer (error 104)
2026-09-26T16:36:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1390 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:36:36 dist-test-slave-2zvq krb5kdc[648](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:36:36 dist-test-slave-2zvq krb5kdc[648](info): set up 2 sockets
Sep 26 16:36:36 dist-test-slave-2zvq krb5kdc[648](info): commencing operation
krb5kdc: starting...
W20260926 16:36:37.656224 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:36:37 dist-test-slave-2zvq krb5kdc[648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:36:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:36:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
I20260926 16:36:37.772672 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:45635
--webserver_interface=127.11.8.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:37801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:45635
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:36:37.880205 661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:37.880505 661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:37.880589 661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:37.884915 661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:36:37.885016 661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:37.885057 661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:37.885113 661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:36:37.885152 661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:36:37.890457 661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:37801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:45635
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:45635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:37.891429 661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:37.892407 661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:37.900559 669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:37.900565 667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:37.900656 661 server_base.cc:1034] running on GCE node
W20260926 16:36:37.900583 666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:37.901357 661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:37.902437 661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:37.903632 661 hybrid_clock.cc:648] HybridClock initialized: now 1790440597903601 us; error 43 us; skew 500 ppm
Sep 26 16:36:37 dist-test-slave-2zvq krb5kdc[648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440597, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:36:37.906253 661 init.cc:377] Logged in from keytab as kudu/127.11.8.254@KRBTEST.COM (short username kudu)
I20260926 16:36:37.907090 661 webserver.cc:460] Webserver started at http://127.11.8.254:42471/ using document root <none> and password file <none>
I20260926 16:36:37.907579 661 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:37.907634 661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:37.907837 661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:37.909713 661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "b729aa3150ae4f8e85481a80d23af5e7"
format_stamp: "Formatted at 2026-09-26 16:36:37 on dist-test-slave-2zvq"
server_key: "ae0f560b320ce128a8f02b5eb3f6d27e"
server_key_iv: "8eb3252c955fbe72a22987beaf308a7c"
server_key_version: "encryptionkey@0"
I20260926 16:36:37.910214 661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "b729aa3150ae4f8e85481a80d23af5e7"
format_stamp: "Formatted at 2026-09-26 16:36:37 on dist-test-slave-2zvq"
server_key: "ae0f560b320ce128a8f02b5eb3f6d27e"
server_key_iv: "8eb3252c955fbe72a22987beaf308a7c"
server_key_version: "encryptionkey@0"
I20260926 16:36:37.913661 661 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260926 16:36:37.915931 676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:37.917083 661 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260926 16:36:37.917201 661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "b729aa3150ae4f8e85481a80d23af5e7"
format_stamp: "Formatted at 2026-09-26 16:36:37 on dist-test-slave-2zvq"
server_key: "ae0f560b320ce128a8f02b5eb3f6d27e"
server_key_iv: "8eb3252c955fbe72a22987beaf308a7c"
server_key_version: "encryptionkey@0"
I20260926 16:36:37.917296 661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:37.921675 661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:37.924341 661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:37.924515 661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:37.931129 661 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:45635
I20260926 16:36:37.931140 728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:45635 every 8 connection(s)
I20260926 16:36:37.931986 661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:36:37.934580 729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:37.938383 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 661
I20260926 16:36:37.938512 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:36:37.938771 11299 external_mini_cluster.cc:1428] Setting key 84257c211826cb0282da017499dcf854
I20260926 16:36:37.940151 729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b729aa3150ae4f8e85481a80d23af5e7: Bootstrap starting.
I20260926 16:36:37.942363 729 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b729aa3150ae4f8e85481a80d23af5e7: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:37.943066 729 log.cc:824] T 00000000000000000000000000000000 P b729aa3150ae4f8e85481a80d23af5e7: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:37.943885 732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:37.940174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60041 (local address 127.11.8.254:45635)
0926 16:36:37.940580 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:37.940594 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:37.940628 (+ 34us) server_negotiation.cc:407] Connection header received
0926 16:36:37.941314 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:37.941330 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:37.941597 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:37.941923 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:37.942762 (+ 839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:37.942767 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:37.942786 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:36:37.942796 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:37.942851 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:36:37.942870 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:37.942978 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:37.943079 (+ 101us) server_negotiation.cc:299] Negotiation successful
0926 16:36:37.943229 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":116,"threads_started":1}
I20260926 16:36:37.944931 729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b729aa3150ae4f8e85481a80d23af5e7: No bootstrap required, opened a new log
I20260926 16:36:37.946631 11299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:36:37.948889 729 raft_consensus.cc:348] T 00000000000000000000000000000000 P b729aa3150ae4f8e85481a80d23af5e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b729aa3150ae4f8e85481a80d23af5e7" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 45635 } }
I20260926 16:36:37.949111 729 raft_consensus.cc:374] T 00000000000000000000000000000000 P b729aa3150ae4f8e85481a80d23af5e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:36:37.949170 729 raft_consensus.cc:729] T 00000000000000000000000000000000 P b729aa3150ae4f8e85481a80d23af5e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b729aa3150ae4f8e85481a80d23af5e7, State: Initialized, Role: FOLLOWER
I20260926 16:36:37.949631 729 consensus_queue.cc:260] T 00000000000000000000000000000000 P b729aa3150ae4f8e85481a80d23af5e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b729aa3150ae4f8e85481a80d23af5e7" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 45635 } }
I20260926 16:36:37.949785 729 raft_consensus.cc:388] T 00000000000000000000000000000000 P b729aa3150ae4f8e85481a80d23af5e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:36:37.949874 729 raft_consensus.cc:482] T 00000000000000000000000000000000 P b729aa3150ae4f8e85481a80d23af5e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:36:37.949988 729 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b729aa3150ae4f8e85481a80d23af5e7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:37.950140 732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:37.948502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41547 (local address 127.11.8.254:45635)
0926 16:36:37.948598 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:37.948601 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:37.948801 (+ 200us) server_negotiation.cc:407] Connection header received
0926 16:36:37.948937 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:37.948940 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:37.948969 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:37.949051 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:37.949695 (+ 644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:37.949698 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:37.949701 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:36:37.949711 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:37.949735 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:36:37.949747 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:37.949840 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:37.949965 (+ 125us) server_negotiation.cc:299] Negotiation successful
0926 16:36:37.949999 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 16:36:37.951124 729 raft_consensus.cc:504] T 00000000000000000000000000000000 P b729aa3150ae4f8e85481a80d23af5e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b729aa3150ae4f8e85481a80d23af5e7" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 45635 } }
I20260926 16:36:37.951447 729 leader_election.cc:302] T 00000000000000000000000000000000 P b729aa3150ae4f8e85481a80d23af5e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b729aa3150ae4f8e85481a80d23af5e7; no voters:
I20260926 16:36:37.951713 729 leader_election.cc:288] T 00000000000000000000000000000000 P b729aa3150ae4f8e85481a80d23af5e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:36:37.951787 739 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b729aa3150ae4f8e85481a80d23af5e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:36:37.951992 739 raft_consensus.cc:686] T 00000000000000000000000000000000 P b729aa3150ae4f8e85481a80d23af5e7 [term 1 LEADER]: Becoming Leader. State: Replica: b729aa3150ae4f8e85481a80d23af5e7, State: Running, Role: LEADER
I20260926 16:36:37.952304 739 consensus_queue.cc:237] T 00000000000000000000000000000000 P b729aa3150ae4f8e85481a80d23af5e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b729aa3150ae4f8e85481a80d23af5e7" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 45635 } }
I20260926 16:36:37.952821 729 sys_catalog.cc:564] T 00000000000000000000000000000000 P b729aa3150ae4f8e85481a80d23af5e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:36:37.953867 741 sys_catalog.cc:455] T 00000000000000000000000000000000 P b729aa3150ae4f8e85481a80d23af5e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b729aa3150ae4f8e85481a80d23af5e7. Latest consensus state: current_term: 1 leader_uuid: "b729aa3150ae4f8e85481a80d23af5e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b729aa3150ae4f8e85481a80d23af5e7" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 45635 } } }
I20260926 16:36:37.953999 741 sys_catalog.cc:458] T 00000000000000000000000000000000 P b729aa3150ae4f8e85481a80d23af5e7 [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:37.954391 748 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:36:37.954866 740 sys_catalog.cc:455] T 00000000000000000000000000000000 P b729aa3150ae4f8e85481a80d23af5e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b729aa3150ae4f8e85481a80d23af5e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b729aa3150ae4f8e85481a80d23af5e7" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 45635 } } }
I20260926 16:36:37.954960 740 sys_catalog.cc:458] T 00000000000000000000000000000000 P b729aa3150ae4f8e85481a80d23af5e7 [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:37.956908 748 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:36:37.961771 748 catalog_manager.cc:1347] Generated new cluster ID: 6d3e15a882d14761bd9a2642f93782b4
I20260926 16:36:37.961830 748 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:36:37.975605 748 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:36:37.975895 748 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:36:37.985548 748 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b729aa3150ae4f8e85481a80d23af5e7: Generated new TSK 0
I20260926 16:36:37.986228 748 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:36:37.991695 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 661
2026-09-26T16:36:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1371 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:36:38 dist-test-slave-2zvq krb5kdc[760](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:36:38 dist-test-slave-2zvq krb5kdc[760](info): set up 2 sockets
Sep 26 16:36:38 dist-test-slave-2zvq krb5kdc[760](info): commencing operation
krb5kdc: starting...
W20260926 16:36:39.020231 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.009s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:36:39 dist-test-slave-2zvq krb5kdc[760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:36:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:36:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
I20260926 16:36:39.137530 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:37305
--webserver_interface=127.11.8.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:43925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:37305
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:36:39.245298 773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:39.245587 773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:39.245661 773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:39.249572 773 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:36:39.249657 773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:39.249696 773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:39.249749 773 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:36:39.249788 773 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:36:39.254838 773 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:43925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:37305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:37305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:39.255824 773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:39.256824 773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:39.264925 779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:39.265057 773 server_base.cc:1034] running on GCE node
W20260926 16:36:39.264935 781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:39.264981 778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:39.265733 773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:39.266788 773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:39.267966 773 hybrid_clock.cc:648] HybridClock initialized: now 1790440599267949 us; error 25 us; skew 500 ppm
Sep 26 16:36:39 dist-test-slave-2zvq krb5kdc[760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440599, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:36:39.276021 773 init.cc:377] Logged in from keytab as kudu/127.11.8.254@KRBTEST.COM (short username kudu)
I20260926 16:36:39.276891 773 webserver.cc:460] Webserver started at http://127.11.8.254:42799/ using document root <none> and password file <none>
I20260926 16:36:39.277613 773 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:39.277814 773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:39.278051 773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:39.280115 773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "9ccceebcd18c4be2893fffa2457aba64"
format_stamp: "Formatted at 2026-09-26 16:36:39 on dist-test-slave-2zvq"
server_key: "c26a2bd20a13553593db8c6527f42a5a"
server_key_iv: "d4e6138db5c727c0b8dbcb658ed71732"
server_key_version: "encryptionkey@0"
I20260926 16:36:39.280673 773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "9ccceebcd18c4be2893fffa2457aba64"
format_stamp: "Formatted at 2026-09-26 16:36:39 on dist-test-slave-2zvq"
server_key: "c26a2bd20a13553593db8c6527f42a5a"
server_key_iv: "d4e6138db5c727c0b8dbcb658ed71732"
server_key_version: "encryptionkey@0"
I20260926 16:36:39.284369 773 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 16:36:39.286863 789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:39.288053 773 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:36:39.288188 773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "9ccceebcd18c4be2893fffa2457aba64"
format_stamp: "Formatted at 2026-09-26 16:36:39 on dist-test-slave-2zvq"
server_key: "c26a2bd20a13553593db8c6527f42a5a"
server_key_iv: "d4e6138db5c727c0b8dbcb658ed71732"
server_key_version: "encryptionkey@0"
I20260926 16:36:39.288290 773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:39.313191 773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:39.316131 773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:39.316329 773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:39.323380 773 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:37305
I20260926 16:36:39.323402 844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:37305 every 8 connection(s)
I20260926 16:36:39.324302 773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:36:39.327040 845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:39.332362 845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ccceebcd18c4be2893fffa2457aba64: Bootstrap starting.
I20260926 16:36:39.333501 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 773
I20260926 16:36:39.333639 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:36:39.333914 11299 external_mini_cluster.cc:1428] Setting key e84001f820397f1fb9f1a64f0dde0070
I20260926 16:36:39.335173 845 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ccceebcd18c4be2893fffa2457aba64: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:39.335886 845 log.cc:824] T 00000000000000000000000000000000 P 9ccceebcd18c4be2893fffa2457aba64: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:39.337733 845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ccceebcd18c4be2893fffa2457aba64: No bootstrap required, opened a new log
I20260926 16:36:39.341295 845 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ccceebcd18c4be2893fffa2457aba64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ccceebcd18c4be2893fffa2457aba64" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37305 } }
I20260926 16:36:39.341571 845 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ccceebcd18c4be2893fffa2457aba64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:36:39.341634 845 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ccceebcd18c4be2893fffa2457aba64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ccceebcd18c4be2893fffa2457aba64, State: Initialized, Role: FOLLOWER
I20260926 16:36:39.342125 845 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ccceebcd18c4be2893fffa2457aba64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ccceebcd18c4be2893fffa2457aba64" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37305 } }
I20260926 16:36:39.342317 845 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ccceebcd18c4be2893fffa2457aba64 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:36:39.342392 845 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ccceebcd18c4be2893fffa2457aba64 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:36:39.342509 845 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ccceebcd18c4be2893fffa2457aba64 [term 0 FOLLOWER]: Advancing to term 1
Sep 26 16:36:39 dist-test-slave-2zvq krb5kdc[760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:36:39.343340 845 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ccceebcd18c4be2893fffa2457aba64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ccceebcd18c4be2893fffa2457aba64" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37305 } }
I20260926 16:36:39.343673 845 leader_election.cc:302] T 00000000000000000000000000000000 P 9ccceebcd18c4be2893fffa2457aba64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ccceebcd18c4be2893fffa2457aba64; no voters:
I20260926 16:36:39.344012 845 leader_election.cc:288] T 00000000000000000000000000000000 P 9ccceebcd18c4be2893fffa2457aba64 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:36:39.344300 850 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ccceebcd18c4be2893fffa2457aba64 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:36:39.344516 850 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ccceebcd18c4be2893fffa2457aba64 [term 1 LEADER]: Becoming Leader. State: Replica: 9ccceebcd18c4be2893fffa2457aba64, State: Running, Role: LEADER
I20260926 16:36:39.344805 850 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ccceebcd18c4be2893fffa2457aba64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ccceebcd18c4be2893fffa2457aba64" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37305 } }
I20260926 16:36:39.345208 845 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ccceebcd18c4be2893fffa2457aba64 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:36:39.346521 851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ccceebcd18c4be2893fffa2457aba64 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ccceebcd18c4be2893fffa2457aba64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ccceebcd18c4be2893fffa2457aba64" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37305 } } }
I20260926 16:36:39.346531 852 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ccceebcd18c4be2893fffa2457aba64 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ccceebcd18c4be2893fffa2457aba64. Latest consensus state: current_term: 1 leader_uuid: "9ccceebcd18c4be2893fffa2457aba64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ccceebcd18c4be2893fffa2457aba64" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37305 } } }
I20260926 16:36:39.346648 851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ccceebcd18c4be2893fffa2457aba64 [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:39.346659 852 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ccceebcd18c4be2893fffa2457aba64 [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:39.347049 862 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:36:39.349695 848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:39.334979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53207 (local address 127.11.8.254:37305)
0926 16:36:39.335460 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:39.335474 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:39.335509 (+ 35us) server_negotiation.cc:407] Connection header received
0926 16:36:39.336171 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:39.336205 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:39.336633 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:39.336957 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:39.337738 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:39.338798 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:39.339646 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:39.340105 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:39.340272 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:39.343194 (+ 2922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:39.343204 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:39.343216 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:39.343241 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:39.345163 (+ 1922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:39.345617 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:39.345625 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:39.345632 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:39.345775 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:39.346245 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:39.346264 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:39.346267 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:39.347541 (+ 1274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:39.347741 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:39.348509 (+ 768us) server_negotiation.cc:299] Negotiation successful
0926 16:36:39.348793 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":150,"threads_started":1}
I20260926 16:36:39.349727 862 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:36:39.352555 11299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:36:39.355443 862 catalog_manager.cc:1347] Generated new cluster ID: 68d8960d410044219fecebcc7934f0ff
I20260926 16:36:39.355504 862 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:36:39.362814 848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:39.354438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46995 (local address 127.11.8.254:37305)
0926 16:36:39.354563 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:39.354568 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:39.354734 (+ 166us) server_negotiation.cc:407] Connection header received
0926 16:36:39.354871 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:39.354874 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:39.354914 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:39.354996 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:39.355766 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:39.356572 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:39.357653 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:39.358001 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:39.358109 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:39.359537 (+ 1428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:39.359541 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:39.359543 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:39.359568 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:39.361126 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:39.361548 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:39.361552 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:39.361553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:39.361636 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:39.361961 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:39.361966 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:39.361969 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:39.362166 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:39.362321 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:39.362559 (+ 238us) server_negotiation.cc:299] Negotiation successful
0926 16:36:39.362669 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260926 16:36:39.375515 862 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:36:39.376667 862 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:36:39.386755 862 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ccceebcd18c4be2893fffa2457aba64: Generated new TSK 0
I20260926 16:36:39.387573 862 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:36:39.391345 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 773
2026-09-26T16:36:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1399 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:36:39 dist-test-slave-2zvq krb5kdc[877](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:36:39 dist-test-slave-2zvq krb5kdc[877](info): set up 2 sockets
Sep 26 16:36:39 dist-test-slave-2zvq krb5kdc[877](info): commencing operation
krb5kdc: starting...
W20260926 16:36:40.428392 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:36:40 dist-test-slave-2zvq krb5kdc[877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:36:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:36:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
I20260926 16:36:40.551133 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:44121
--webserver_interface=127.11.8.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:40833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:44121
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:36:40.659431 890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:40.659693 890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:40.659734 890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:40.664211 890 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:36:40.664270 890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:40.664289 890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:40.664314 890 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:36:40.664330 890 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:36:40.669371 890 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:40833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:44121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:44121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:40.670356 890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:40.671303 890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:40.679673 896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:40.679713 897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:40.679670 899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:40.679785 890 server_base.cc:1034] running on GCE node
I20260926 16:36:40.680521 890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:40.681725 890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:40.682974 890 hybrid_clock.cc:648] HybridClock initialized: now 1790440600682941 us; error 44 us; skew 500 ppm
Sep 26 16:36:40 dist-test-slave-2zvq krb5kdc[877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440600, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:36:40.685760 890 init.cc:377] Logged in from keytab as kudu/127.11.8.254@KRBTEST.COM (short username kudu)
I20260926 16:36:40.686718 890 webserver.cc:460] Webserver started at http://127.11.8.254:36653/ using document root <none> and password file <none>
I20260926 16:36:40.687258 890 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:40.687318 890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:40.687525 890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:40.689565 890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "36f700c59c27410a970899b5b5d57417"
format_stamp: "Formatted at 2026-09-26 16:36:40 on dist-test-slave-2zvq"
server_key: "78ef35c4f76d8cd0bc24a4351c1d6b59"
server_key_iv: "8fb6f281238114af2a06804859a55644"
server_key_version: "encryptionkey@0"
I20260926 16:36:40.690117 890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "36f700c59c27410a970899b5b5d57417"
format_stamp: "Formatted at 2026-09-26 16:36:40 on dist-test-slave-2zvq"
server_key: "78ef35c4f76d8cd0bc24a4351c1d6b59"
server_key_iv: "8fb6f281238114af2a06804859a55644"
server_key_version: "encryptionkey@0"
I20260926 16:36:40.693681 890 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260926 16:36:40.696046 906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:40.697227 890 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:36:40.697357 890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "36f700c59c27410a970899b5b5d57417"
format_stamp: "Formatted at 2026-09-26 16:36:40 on dist-test-slave-2zvq"
server_key: "78ef35c4f76d8cd0bc24a4351c1d6b59"
server_key_iv: "8fb6f281238114af2a06804859a55644"
server_key_version: "encryptionkey@0"
I20260926 16:36:40.697460 890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:40.715688 890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:40.719045 890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:40.719257 890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:40.726042 890 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:44121
I20260926 16:36:40.726106 961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:44121 every 8 connection(s)
I20260926 16:36:40.727048 890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:36:40.729786 962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:40.735213 962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36f700c59c27410a970899b5b5d57417: Bootstrap starting.
I20260926 16:36:40.737015 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 890
I20260926 16:36:40.737164 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:36:40.737461 11299 external_mini_cluster.cc:1428] Setting key 52c51feedd47a6fa960e8e1f36374173
I20260926 16:36:40.737955 962 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36f700c59c27410a970899b5b5d57417: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:40.738867 962 log.cc:824] T 00000000000000000000000000000000 P 36f700c59c27410a970899b5b5d57417: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:40.740693 962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36f700c59c27410a970899b5b5d57417: No bootstrap required, opened a new log
I20260926 16:36:40.744086 962 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36f700c59c27410a970899b5b5d57417 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f700c59c27410a970899b5b5d57417" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 44121 } }
I20260926 16:36:40.744315 962 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36f700c59c27410a970899b5b5d57417 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:36:40.744390 962 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36f700c59c27410a970899b5b5d57417 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36f700c59c27410a970899b5b5d57417, State: Initialized, Role: FOLLOWER
I20260926 16:36:40.744838 962 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36f700c59c27410a970899b5b5d57417 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f700c59c27410a970899b5b5d57417" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 44121 } }
I20260926 16:36:40.744987 962 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36f700c59c27410a970899b5b5d57417 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:36:40.745054 962 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36f700c59c27410a970899b5b5d57417 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:36:40.745152 962 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36f700c59c27410a970899b5b5d57417 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:40.745980 965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:40.738684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54961 (local address 127.11.8.254:44121)
0926 16:36:40.739163 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:40.739181 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:40.739227 (+ 46us) server_negotiation.cc:407] Connection header received
0926 16:36:40.739895 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:40.739913 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:40.740239 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:40.740582 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:40.740987 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:40.741913 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:40.742838 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:40.743307 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:40.743485 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:40.744275 (+ 790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:40.744281 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:40.744293 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:36:40.744762 (+ 469us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:40.744815 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:36:40.744839 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:40.744948 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:40.745060 (+ 112us) server_negotiation.cc:299] Negotiation successful
0926 16:36:40.745271 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":138,"threads_started":1}
I20260926 16:36:40.746003 962 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36f700c59c27410a970899b5b5d57417 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f700c59c27410a970899b5b5d57417" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 44121 } }
I20260926 16:36:40.746397 962 leader_election.cc:302] T 00000000000000000000000000000000 P 36f700c59c27410a970899b5b5d57417 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36f700c59c27410a970899b5b5d57417; no voters:
I20260926 16:36:40.746717 962 leader_election.cc:288] T 00000000000000000000000000000000 P 36f700c59c27410a970899b5b5d57417 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:36:40.746783 967 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36f700c59c27410a970899b5b5d57417 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:36:40.746994 967 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36f700c59c27410a970899b5b5d57417 [term 1 LEADER]: Becoming Leader. State: Replica: 36f700c59c27410a970899b5b5d57417, State: Running, Role: LEADER
I20260926 16:36:40.747344 967 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36f700c59c27410a970899b5b5d57417 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f700c59c27410a970899b5b5d57417" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 44121 } }
I20260926 16:36:40.748060 962 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36f700c59c27410a970899b5b5d57417 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:36:40.748925 11299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:36:40.748986 968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36f700c59c27410a970899b5b5d57417 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36f700c59c27410a970899b5b5d57417. Latest consensus state: current_term: 1 leader_uuid: "36f700c59c27410a970899b5b5d57417" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f700c59c27410a970899b5b5d57417" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 44121 } } }
I20260926 16:36:40.749086 968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36f700c59c27410a970899b5b5d57417 [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:40.749328 969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36f700c59c27410a970899b5b5d57417 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36f700c59c27410a970899b5b5d57417" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f700c59c27410a970899b5b5d57417" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 44121 } } }
I20260926 16:36:40.749413 969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36f700c59c27410a970899b5b5d57417 [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:40.749665 976 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:36:40.752646 976 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:36:40.756995 965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:40.751263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52503 (local address 127.11.8.254:44121)
0926 16:36:40.751369 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:40.751373 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:40.751763 (+ 390us) server_negotiation.cc:407] Connection header received
0926 16:36:40.752021 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:40.752025 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:40.752067 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:40.752116 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:40.752968 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:40.753729 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:40.754930 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:40.755244 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:40.755364 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:40.756253 (+ 889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:40.756256 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:40.756258 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:36:40.756270 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:40.756294 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:36:40.756304 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:40.756392 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:40.756801 (+ 409us) server_negotiation.cc:299] Negotiation successful
0926 16:36:40.756859 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260926 16:36:40.758008 976 catalog_manager.cc:1347] Generated new cluster ID: 624a73486310463789a869daebe167c7
I20260926 16:36:40.758064 976 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:36:40.770741 976 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:36:40.771947 976 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:36:40.789076 976 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36f700c59c27410a970899b5b5d57417: Generated new TSK 0
I20260926 16:36:40.790063 976 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:36:40.800843 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 890
2026-09-26T16:36:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1409 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:36:40 dist-test-slave-2zvq krb5kdc[993](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:36:40 dist-test-slave-2zvq krb5kdc[993](info): set up 2 sockets
Sep 26 16:36:40 dist-test-slave-2zvq krb5kdc[993](info): commencing operation
krb5kdc: starting...
W20260926 16:36:41.824333 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.004s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:36:41 dist-test-slave-2zvq krb5kdc[993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:36:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:36:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
I20260926 16:36:41.944694 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:37219
--webserver_interface=127.11.8.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:40543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:37219
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:36:42.053212 1006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:42.053480 1006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:42.053521 1006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:42.057456 1006 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:36:42.057516 1006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:42.057536 1006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:42.057554 1006 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:36:42.057569 1006 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:36:42.062660 1006 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:40543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:37219
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:37219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:42.063591 1006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:42.064525 1006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:42.072721 1014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:42.072731 1011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:42.072782 1006 server_base.cc:1034] running on GCE node
W20260926 16:36:42.072731 1012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:42.073647 1006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:42.074681 1006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:42.075840 1006 hybrid_clock.cc:648] HybridClock initialized: now 1790440602075823 us; error 29 us; skew 500 ppm
Sep 26 16:36:42 dist-test-slave-2zvq krb5kdc[993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440602, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:36:42.078495 1006 init.cc:377] Logged in from keytab as kudu/127.11.8.254@KRBTEST.COM (short username kudu)
I20260926 16:36:42.079349 1006 webserver.cc:460] Webserver started at http://127.11.8.254:40001/ using document root <none> and password file <none>
I20260926 16:36:42.079916 1006 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:42.079980 1006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:42.080186 1006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:42.082158 1006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "c7489037eb644c43952880be1eaa8492"
format_stamp: "Formatted at 2026-09-26 16:36:42 on dist-test-slave-2zvq"
server_key: "ae0280b369942b72a8c0372a2f24e896"
server_key_iv: "5658674d5bf97ae536081e2e1ae42d91"
server_key_version: "encryptionkey@0"
I20260926 16:36:42.082782 1006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "c7489037eb644c43952880be1eaa8492"
format_stamp: "Formatted at 2026-09-26 16:36:42 on dist-test-slave-2zvq"
server_key: "ae0280b369942b72a8c0372a2f24e896"
server_key_iv: "5658674d5bf97ae536081e2e1ae42d91"
server_key_version: "encryptionkey@0"
I20260926 16:36:42.086326 1006 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 16:36:42.088683 1021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:42.089840 1006 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:36:42.089967 1006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "c7489037eb644c43952880be1eaa8492"
format_stamp: "Formatted at 2026-09-26 16:36:42 on dist-test-slave-2zvq"
server_key: "ae0280b369942b72a8c0372a2f24e896"
server_key_iv: "5658674d5bf97ae536081e2e1ae42d91"
server_key_version: "encryptionkey@0"
I20260926 16:36:42.090068 1006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:42.096117 1006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:42.100023 1006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:42.100167 1006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:42.107029 1006 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:37219
I20260926 16:36:42.107036 1074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:37219 every 8 connection(s)
I20260926 16:36:42.107928 1006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:36:42.110060 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 1006
I20260926 16:36:42.110200 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:36:42.110531 11299 external_mini_cluster.cc:1428] Setting key 8428aa9943be015882ea1d00050ec2bc
I20260926 16:36:42.110802 1075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:42.115573 1078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:42.111882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34727 (local address 127.11.8.254:37219)
0926 16:36:42.112309 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:42.112324 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:42.112357 (+ 33us) server_negotiation.cc:407] Connection header received
0926 16:36:42.112984 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:42.113001 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:42.113267 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:42.113590 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:42.114453 (+ 863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:42.114458 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:42.114469 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:36:42.114480 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:42.114531 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:36:42.114550 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:42.114693 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:42.114763 (+ 70us) server_negotiation.cc:299] Negotiation successful
0926 16:36:42.114908 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":145,"threads_started":1}
I20260926 16:36:42.117501 1075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7489037eb644c43952880be1eaa8492: Bootstrap starting.
I20260926 16:36:42.118323 11299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:36:42.119907 1075 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7489037eb644c43952880be1eaa8492: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:42.120757 1075 log.cc:824] T 00000000000000000000000000000000 P c7489037eb644c43952880be1eaa8492: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:42.121819 1078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:42.120295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42687 (local address 127.11.8.254:37219)
0926 16:36:42.120408 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:42.120412 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:42.120591 (+ 179us) server_negotiation.cc:407] Connection header received
0926 16:36:42.120729 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:42.120731 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:42.120757 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:42.120813 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:42.121461 (+ 648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:42.121467 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:42.121473 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 16:36:42.121489 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:42.121516 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 16:36:42.121529 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:42.121625 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:42.121660 (+ 35us) server_negotiation.cc:299] Negotiation successful
0926 16:36:42.121699 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 16:36:42.123139 1075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7489037eb644c43952880be1eaa8492: No bootstrap required, opened a new log
I20260926 16:36:42.126821 1075 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7489037eb644c43952880be1eaa8492 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7489037eb644c43952880be1eaa8492" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37219 } }
I20260926 16:36:42.127066 1075 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7489037eb644c43952880be1eaa8492 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 16:36:42.127146 1075 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7489037eb644c43952880be1eaa8492 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7489037eb644c43952880be1eaa8492, State: Initialized, Role: FOLLOWER
I20260926 16:36:42.127530 1075 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7489037eb644c43952880be1eaa8492 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7489037eb644c43952880be1eaa8492" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37219 } }
I20260926 16:36:42.127676 1075 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7489037eb644c43952880be1eaa8492 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 16:36:42.127760 1075 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7489037eb644c43952880be1eaa8492 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 16:36:42.127868 1075 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7489037eb644c43952880be1eaa8492 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:42.128722 1075 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7489037eb644c43952880be1eaa8492 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7489037eb644c43952880be1eaa8492" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37219 } }
I20260926 16:36:42.129053 1075 leader_election.cc:302] T 00000000000000000000000000000000 P c7489037eb644c43952880be1eaa8492 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7489037eb644c43952880be1eaa8492; no voters:
I20260926 16:36:42.129343 1075 leader_election.cc:288] T 00000000000000000000000000000000 P c7489037eb644c43952880be1eaa8492 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 16:36:42.129456 1085 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7489037eb644c43952880be1eaa8492 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:36:42.129716 1085 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7489037eb644c43952880be1eaa8492 [term 1 LEADER]: Becoming Leader. State: Replica: c7489037eb644c43952880be1eaa8492, State: Running, Role: LEADER
I20260926 16:36:42.130075 1085 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7489037eb644c43952880be1eaa8492 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7489037eb644c43952880be1eaa8492" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37219 } }
I20260926 16:36:42.130586 1075 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7489037eb644c43952880be1eaa8492 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:36:42.131961 1086 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7489037eb644c43952880be1eaa8492 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7489037eb644c43952880be1eaa8492" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7489037eb644c43952880be1eaa8492" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37219 } } }
I20260926 16:36:42.131963 1087 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7489037eb644c43952880be1eaa8492 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7489037eb644c43952880be1eaa8492. Latest consensus state: current_term: 1 leader_uuid: "c7489037eb644c43952880be1eaa8492" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7489037eb644c43952880be1eaa8492" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37219 } } }
I20260926 16:36:42.132107 1086 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7489037eb644c43952880be1eaa8492 [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:42.132113 1087 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7489037eb644c43952880be1eaa8492 [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:42.132496 1090 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:36:42.135653 1090 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:36:42.141539 1090 catalog_manager.cc:1347] Generated new cluster ID: 559bbcb40eaf44ea92a4aeae924eec5b
I20260926 16:36:42.141631 1090 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:36:42.153229 1090 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:36:42.153518 1090 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:36:42.159183 1090 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7489037eb644c43952880be1eaa8492: Generated new TSK 0
I20260926 16:36:42.159807 1090 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 16:36:42.171330 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 1006
2026-09-26T16:36:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1370 ms)
[----------] 18 tests from AuthTokenIssuingTest (25254 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:36:42 dist-test-slave-2zvq krb5kdc[1106](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:36:42 dist-test-slave-2zvq krb5kdc[1106](info): set up 2 sockets
Sep 26 16:36:42 dist-test-slave-2zvq krb5kdc[1106](info): commencing operation
krb5kdc: starting...
W20260926 16:36:43.208244 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:36:43 dist-test-slave-2zvq krb5kdc[1106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:36:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:36:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
I20260926 16:36:43.327001 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:44201
--webserver_interface=127.11.8.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:35833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:44201,127.11.8.253:41685,127.11.8.252:36607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:36:43.435839 1119 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 16:36:43.436180 1119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:43.436244 1119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:43.436329 1119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:43.440366 1119 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:36:43.440440 1119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:43.440477 1119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:43.440526 1119 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:36:43.440562 1119 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:36:43.445724 1119 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:35833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:44201,127.11.8.253:41685,127.11.8.252:36607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:44201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:43.446857 1119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:43.447870 1119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:43.456288 1124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:43.456269 1125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:43.456411 1119 server_base.cc:1034] running on GCE node
W20260926 16:36:43.456269 1127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:43.457147 1119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:43.458185 1119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:43.459388 1119 hybrid_clock.cc:648] HybridClock initialized: now 1790440603459367 us; error 30 us; skew 500 ppm
Sep 26 16:36:43 dist-test-slave-2zvq krb5kdc[1106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440603, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:36:43.462045 1119 init.cc:377] Logged in from keytab as kudu/127.11.8.254@KRBTEST.COM (short username kudu)
I20260926 16:36:43.462975 1119 webserver.cc:460] Webserver started at http://127.11.8.254:43213/ using document root <none> and password file <none>
I20260926 16:36:43.463505 1119 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:43.463591 1119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:43.463815 1119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:43.465734 1119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "faca929a73f642f1b760ff7467020f53"
format_stamp: "Formatted at 2026-09-26 16:36:43 on dist-test-slave-2zvq"
server_key: "0abcafe070bd18466baa86fa22677bde"
server_key_iv: "f37dfb2312fe3a52ebec2d0c9c7dacf2"
server_key_version: "encryptionkey@0"
I20260926 16:36:43.466405 1119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "faca929a73f642f1b760ff7467020f53"
format_stamp: "Formatted at 2026-09-26 16:36:43 on dist-test-slave-2zvq"
server_key: "0abcafe070bd18466baa86fa22677bde"
server_key_iv: "f37dfb2312fe3a52ebec2d0c9c7dacf2"
server_key_version: "encryptionkey@0"
I20260926 16:36:43.469848 1119 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260926 16:36:43.472211 1134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:43.473438 1119 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 16:36:43.473577 1119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "faca929a73f642f1b760ff7467020f53"
format_stamp: "Formatted at 2026-09-26 16:36:43 on dist-test-slave-2zvq"
server_key: "0abcafe070bd18466baa86fa22677bde"
server_key_iv: "f37dfb2312fe3a52ebec2d0c9c7dacf2"
server_key_version: "encryptionkey@0"
I20260926 16:36:43.473682 1119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:43.496719 1119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:43.499634 1119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:43.499836 1119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:43.507061 1119 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:44201
I20260926 16:36:43.507408 1187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:44201 every 8 connection(s)
I20260926 16:36:43.508050 1119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:36:43.510859 1188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:43.513150 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 1119
I20260926 16:36:43.513345 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:36:43.513610 11299 external_mini_cluster.cc:1428] Setting key 209685ca5a97326c4180acd0084d51f4
I20260926 16:36:43.514648 1188 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.254" port: 44201 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.11.8.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.253@KRBTEST.COM" created.
Sep 26 16:36:43 dist-test-slave-2zvq krb5kdc[1106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440603, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:36:43.529206 1192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:43.516856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:44201 (local address 127.0.0.1:36502)
0926 16:36:43.517433 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:43.517468 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:43.518317 (+ 849us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:43.518974 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:43.518979 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:43.519439 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:36:43.519703 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:43.519715 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:43.520801 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:43.520804 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:43.521570 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:43.521579 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:43.522213 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:43.522216 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:43.522351 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:43.522967 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:36:43.522983 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:36:43.524441 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:36:43.526342 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:43.526352 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:43.526362 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:43.526585 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:43.526900 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:43.526905 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:43.526907 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:43.527016 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:43.527569 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:36:43.527575 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:36:43.527709 (+ 134us) client_negotiation.cc:769] Sending connection context
0926 16:36:43.527871 (+ 162us) client_negotiation.cc:240] Negotiation successful
0926 16:36:43.528240 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"mutex_wait_us":103,"spinlock_wait_cycles":29952,"thread_start_us":150,"threads_started":1}
I20260926 16:36:43.529206 1191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:43.516855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36502 (local address 127.11.8.254:44201)
0926 16:36:43.517293 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:43.517312 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:43.517576 (+ 264us) server_negotiation.cc:407] Connection header received
0926 16:36:43.518623 (+ 1047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:43.518627 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:43.518794 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:43.518988 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:43.519842 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:43.520680 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:43.521709 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:43.522062 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:43.522218 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:43.524594 (+ 2376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:43.524600 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:43.524609 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:43.524628 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:43.526209 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:43.526696 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:43.526702 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:43.526708 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:43.526810 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:43.527105 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:43.527111 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:43.527113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:43.527450 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:43.527584 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:43.527941 (+ 357us) server_negotiation.cc:299] Negotiation successful
0926 16:36:43.528183 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":11,"server-negotiator.queue_time_us":283,"thread_start_us":132,"threads_started":1}
I20260926 16:36:43.531373 1188 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.253" port: 41685 } has no permanent_uuid. Determining permanent_uuid...
I20260926 16:36:43.532068 1192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:43.531815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.253:41685 (local address 127.0.0.1:57302)
0926 16:36:43.531942 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:43.532010 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.253:41685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260926 16:36:43.532276 1135 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.8.253:41685: connect: Connection refused (error 111)
I20260926 16:36:43.533087 1192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:43.532879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.253:41685 (local address 127.0.0.1:57304)
0926 16:36:43.532985 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:43.533036 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.253:41685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260926 16:36:43.533229 1188 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.253:41685: Network error: Client connection negotiation failed: client connection to 127.11.8.253:41685: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.253.keytab.
Entry for principal kudu/127.11.8.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.253.keytab.
I20260926 16:36:43.537205 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.253:41685
--webserver_interface=127.11.8.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.253.keytab
--principal=kudu/127.11.8.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:35833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:44201,127.11.8.253:41685,127.11.8.252:36607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 16:36:43.582584 1188 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.253" port: 41685 } attempt: 1
I20260926 16:36:43.583559 1192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:43.583168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.253:41685 (local address 127.0.0.1:57310)
0926 16:36:43.583417 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:43.583482 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.253:41685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260926 16:36:43.584316 1192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:43.584117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.253:41685 (local address 127.0.0.1:57316)
0926 16:36:43.584219 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:43.584264 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.253:41685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260926 16:36:43.584517 1188 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.253:41685: Network error: Client connection negotiation failed: client connection to 127.11.8.253:41685: connect: Connection refused (error 111)
W20260926 16:36:43.646193 1195 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 16:36:43.646479 1195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:43.646502 1195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:43.646574 1195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:43.650554 1195 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:36:43.650614 1195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:43.650633 1195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:43.650650 1195 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:36:43.650664 1195 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:36:43.652688 1188 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.253" port: 41685 } attempt: 2
I20260926 16:36:43.653494 1192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:43.653177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.253:41685 (local address 127.0.0.1:57332)
0926 16:36:43.653365 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:43.653421 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.253:41685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260926 16:36:43.654222 1192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:43.654016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.253:41685 (local address 127.0.0.1:57338)
0926 16:36:43.654121 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:43.654170 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.253:41685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260926 16:36:43.654405 1188 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.253:41685: Network error: Client connection negotiation failed: client connection to 127.11.8.253:41685: connect: Connection refused (error 111)
I20260926 16:36:43.656234 1195 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:35833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:44201,127.11.8.253:41685,127.11.8.252:36607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.253.keytab
--principal=kudu/127.11.8.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.253:41685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.8.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:43.657222 1195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:43.658207 1195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:43.666371 1200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:43.666428 1195 server_base.cc:1034] running on GCE node
W20260926 16:36:43.666350 1201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:43.666360 1203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:43.667044 1195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:43.668044 1195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:43.669209 1195 hybrid_clock.cc:648] HybridClock initialized: now 1790440603669192 us; error 28 us; skew 500 ppm
Sep 26 16:36:43 dist-test-slave-2zvq krb5kdc[1106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440603, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:36:43.671947 1195 init.cc:377] Logged in from keytab as kudu/127.11.8.253@KRBTEST.COM (short username kudu)
I20260926 16:36:43.672817 1195 webserver.cc:460] Webserver started at http://127.11.8.253:45531/ using document root <none> and password file <none>
I20260926 16:36:43.673345 1195 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:43.673429 1195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:43.673641 1195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:43.675629 1195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-1/data/instance:
uuid: "1737b62f0bc74b0a97782c30264530aa"
format_stamp: "Formatted at 2026-09-26 16:36:43 on dist-test-slave-2zvq"
server_key: "57a75310e94005bab66de9ba5b5a7a71"
server_key_iv: "1fce8b95b3c3cbe165b90ec72a724f6e"
server_key_version: "encryptionkey@0"
I20260926 16:36:43.676177 1195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-1/wal/instance:
uuid: "1737b62f0bc74b0a97782c30264530aa"
format_stamp: "Formatted at 2026-09-26 16:36:43 on dist-test-slave-2zvq"
server_key: "57a75310e94005bab66de9ba5b5a7a71"
server_key_iv: "1fce8b95b3c3cbe165b90ec72a724f6e"
server_key_version: "encryptionkey@0"
I20260926 16:36:43.679807 1195 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 16:36:43.682477 1210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:43.683691 1195 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:36:43.683832 1195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-1/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-1/wal
uuid: "1737b62f0bc74b0a97782c30264530aa"
format_stamp: "Formatted at 2026-09-26 16:36:43 on dist-test-slave-2zvq"
server_key: "57a75310e94005bab66de9ba5b5a7a71"
server_key_iv: "1fce8b95b3c3cbe165b90ec72a724f6e"
server_key_version: "encryptionkey@0"
I20260926 16:36:43.683943 1195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:43.702984 1195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:43.706228 1195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:43.706458 1195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:43.713729 1195 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.253:41685
I20260926 16:36:43.714023 1262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.253:41685 every 8 connection(s)
I20260926 16:36:43.714794 1195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-1/data/info.pb
I20260926 16:36:43.717450 1263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:43.721189 1263 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.254" port: 44201 } has no permanent_uuid. Determining permanent_uuid...
I20260926 16:36:43.723229 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 1195
I20260926 16:36:43.723366 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-1/wal/instance
I20260926 16:36:43.723637 11299 external_mini_cluster.cc:1428] Setting key 7d8d793ac36a2f909c47c3907170505b
Sep 26 16:36:43 dist-test-slave-2zvq krb5kdc[1106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440603, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.253@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.8.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.252@KRBTEST.COM" created.
I20260926 16:36:43.736486 1191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:43.723187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36512 (local address 127.11.8.254:44201)
0926 16:36:43.723331 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:43.723335 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:43.724424 (+ 1089us) server_negotiation.cc:407] Connection header received
0926 16:36:43.725611 (+ 1187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:43.725617 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:43.725667 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:43.725753 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:43.727216 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:43.728160 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:43.729298 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:43.729729 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:43.729861 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:43.732313 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:43.732316 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:43.732318 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:43.732343 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:43.733857 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:43.734512 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:43.734515 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:43.734516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:43.734631 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:43.735075 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:43.735082 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:43.735087 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:43.735342 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:43.735433 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:43.736210 (+ 777us) server_negotiation.cc:299] Negotiation successful
0926 16:36:43.736327 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 16:36:43.737422 1265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:43.723626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:44201 (local address 127.0.0.1:36512)
0926 16:36:43.724199 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:43.724246 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:43.725349 (+ 1103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:43.725972 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:43.725983 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:43.726554 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:36:43.727050 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:43.727062 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:43.728300 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:43.728303 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:43.729167 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:43.729176 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:43.729896 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:43.729899 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:43.730043 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:43.730701 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:36:43.730717 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:36:43.732134 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:36:43.734021 (+ 1887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:43.734029 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:43.734046 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:43.734412 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:43.734784 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:43.734788 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:43.734792 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:43.734944 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:43.735481 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:36:43.735489 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:36:43.735851 (+ 362us) client_negotiation.cc:769] Sending connection context
0926 16:36:43.736104 (+ 253us) client_negotiation.cc:240] Negotiation successful
0926 16:36:43.736492 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":158,"threads_started":1}
I20260926 16:36:43.738942 1263 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.253" port: 41685 } has no permanent_uuid. Determining permanent_uuid...
I20260926 16:36:43.745587 1188 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.253" port: 41685 } attempt: 3
Sep 26 16:36:43 dist-test-slave-2zvq krb5kdc[1106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440603, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.253@KRBTEST.COM for kudu/127.11.8.253@KRBTEST.COM
I20260926 16:36:43.750030 1265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:43.739562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.253:41685 (local address 127.0.0.1:57350)
0926 16:36:43.739684 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:43.739701 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:43.739771 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:43.740432 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:43.740435 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:43.740615 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:36:43.740702 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:43.740707 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:43.741751 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:43.741757 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:43.742362 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:43.742370 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:43.742936 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:43.742940 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:43.743028 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:43.743552 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:36:43.743562 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:36:43.746215 (+ 2653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:36:43.748369 (+ 2154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:43.748375 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:43.748378 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:43.748565 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:43.748924 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:43.748927 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:43.748928 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:43.749007 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:43.749541 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:36:43.749544 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:36:43.749638 (+ 94us) client_negotiation.cc:769] Sending connection context
0926 16:36:43.749742 (+ 104us) client_negotiation.cc:240] Negotiation successful
0926 16:36:43.749857 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260926 16:36:43.750963 1268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:43.739734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57350 (local address 127.11.8.253:41685)
0926 16:36:43.740074 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:43.740082 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:43.740101 (+ 19us) server_negotiation.cc:407] Connection header received
0926 16:36:43.740148 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:43.740152 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:43.740301 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:43.740494 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:43.740804 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:43.741641 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:43.742469 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:43.742823 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:43.742934 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:43.746390 (+ 3456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:43.746401 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:43.746421 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:43.746450 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:43.748273 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:43.748720 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:43.748727 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:43.748729 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:43.748820 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:43.749118 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:43.749121 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:43.749123 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:43.749404 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:43.749544 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:43.749831 (+ 287us) server_negotiation.cc:299] Negotiation successful
0926 16:36:43.749959 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":156,"threads_started":1}
Sep 26 16:36:43 dist-test-slave-2zvq krb5kdc[1106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440603, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for kudu/127.11.8.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.252.keytab.
Entry for principal kudu/127.11.8.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.252.keytab.
I20260926 16:36:43.753086 1263 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.252" port: 36607 } has no permanent_uuid. Determining permanent_uuid...
I20260926 16:36:43.753207 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.252:36607
--webserver_interface=127.11.8.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.252.keytab
--principal=kudu/127.11.8.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:35833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:44201,127.11.8.253:41685,127.11.8.252:36607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 16:36:43.753770 1265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:43.753547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:36607 (local address 127.0.0.1:41146)
0926 16:36:43.753647 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:43.753718 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.252:36607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260926 16:36:43.753957 1211 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.8.252:36607: connect: Connection refused (error 111)
I20260926 16:36:43.754819 1265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:43.754590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:36607 (local address 127.0.0.1:41152)
0926 16:36:43.754689 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:43.754745 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.252:36607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260926 16:36:43.754987 1263 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.252:36607: Network error: Client connection negotiation failed: client connection to 127.11.8.252:36607: connect: Connection refused (error 111)
I20260926 16:36:43.757197 1192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:43.746131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.253:41685 (local address 127.0.0.1:57366)
0926 16:36:43.746354 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:43.746370 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:43.746455 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:43.746777 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:43.746779 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:43.746983 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:36:43.747137 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:43.747146 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:43.748176 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:43.748179 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:43.749448 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:43.749461 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:43.750135 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:43.750141 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:43.750290 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:43.750738 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:36:43.750750 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:36:43.752084 (+ 1334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:36:43.755731 (+ 3647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:43.755734 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:43.755737 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:43.755942 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:43.756238 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:43.756241 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:43.756243 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:43.756327 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:43.756690 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:36:43.756694 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:36:43.756792 (+ 98us) client_negotiation.cc:769] Sending connection context
0926 16:36:43.756905 (+ 113us) client_negotiation.cc:240] Negotiation successful
0926 16:36:43.757045 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122}
I20260926 16:36:43.757215 1269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:43.746152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57366 (local address 127.11.8.253:41685)
0926 16:36:43.746538 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:43.746547 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:43.746558 (+ 11us) server_negotiation.cc:407] Connection header received
0926 16:36:43.746588 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:43.746593 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:43.746632 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:43.746741 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:43.747248 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:43.748002 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:43.749551 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:43.749959 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:43.750097 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:43.752250 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:43.752256 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:43.752260 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:43.752282 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:43.755594 (+ 3312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:43.756043 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:43.756048 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:43.756049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:43.756139 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:43.756417 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:43.756421 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:43.756423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:43.756589 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:43.756688 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:43.756954 (+ 266us) server_negotiation.cc:299] Negotiation successful
0926 16:36:43.757060 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":314,"thread_start_us":147,"threads_started":1}
I20260926 16:36:43.757848 1188 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.252" port: 36607 } has no permanent_uuid. Determining permanent_uuid...
I20260926 16:36:43.758582 1192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:43.758262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:36607 (local address 127.0.0.1:41156)
0926 16:36:43.758431 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:43.758499 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.252:36607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260926 16:36:43.759372 1192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:43.759038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:36607 (local address 127.0.0.1:41160)
0926 16:36:43.759206 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:43.759292 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.252:36607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260926 16:36:43.759512 1188 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.252:36607: Network error: Client connection negotiation failed: client connection to 127.11.8.252:36607: connect: Connection refused (error 111)
I20260926 16:36:43.790681 1188 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.252" port: 36607 } attempt: 1
I20260926 16:36:43.791585 1192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:43.791250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:36607 (local address 127.0.0.1:41166)
0926 16:36:43.791422 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:43.791494 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.252:36607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260926 16:36:43.792389 1192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:43.792148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:36607 (local address 127.0.0.1:41176)
0926 16:36:43.792272 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:43.792327 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.252:36607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260926 16:36:43.792549 1188 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.252:36607: Network error: Client connection negotiation failed: client connection to 127.11.8.252:36607: connect: Connection refused (error 111)
I20260926 16:36:43.804241 1263 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.252" port: 36607 } attempt: 1
I20260926 16:36:43.805115 1265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:43.804784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:36607 (local address 127.0.0.1:41180)
0926 16:36:43.804961 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:43.805031 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.252:36607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260926 16:36:43.805871 1265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:43.805652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:36607 (local address 127.0.0.1:41186)
0926 16:36:43.805759 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:43.805810 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.252:36607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260926 16:36:43.806007 1263 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.252:36607: Network error: Client connection negotiation failed: client connection to 127.11.8.252:36607: connect: Connection refused (error 111)
W20260926 16:36:43.864043 1271 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 16:36:43.864295 1271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:43.864318 1271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:43.864355 1271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260926 16:36:43.867712 1188 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.252" port: 36607 } attempt: 2
W20260926 16:36:43.868491 1271 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:36:43.868551 1271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:43.868575 1271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
I20260926 16:36:43.868582 1192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:43.868227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:36607 (local address 127.0.0.1:41200)
0926 16:36:43.868387 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:43.868472 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.252:36607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260926 16:36:43.868605 1271 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:36:43.868633 1271 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:36:43.869344 1192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:43.869108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:36607 (local address 127.0.0.1:41208)
0926 16:36:43.869225 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:43.869283 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.252:36607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260926 16:36:43.869491 1188 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.252:36607: Network error: Client connection negotiation failed: client connection to 127.11.8.252:36607: connect: Connection refused (error 111)
I20260926 16:36:43.874038 1271 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:35833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:44201,127.11.8.253:41685,127.11.8.252:36607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.252.keytab
--principal=kudu/127.11.8.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.252:36607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.8.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:43.874166 1263 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.252" port: 36607 } attempt: 2
I20260926 16:36:43.874842 1265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:43.874582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:36607 (local address 127.0.0.1:41222)
0926 16:36:43.874714 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:43.874770 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.252:36607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260926 16:36:43.875264 1271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:43.875615 1265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:43.875371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:36607 (local address 127.0.0.1:41228)
0926 16:36:43.875506 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:43.875557 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.252:36607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260926 16:36:43.875741 1263 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.252:36607: Network error: Client connection negotiation failed: client connection to 127.11.8.252:36607: connect: Connection refused (error 111)
I20260926 16:36:43.876358 1271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:43.884737 1279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:43.884769 1277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:43.884791 1276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:43.884915 1271 server_base.cc:1034] running on GCE node
I20260926 16:36:43.885517 1271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:43.886646 1271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:43.887850 1271 hybrid_clock.cc:648] HybridClock initialized: now 1790440603887820 us; error 49 us; skew 500 ppm
Sep 26 16:36:43 dist-test-slave-2zvq krb5kdc[1106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440603, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:36:43.890749 1271 init.cc:377] Logged in from keytab as kudu/127.11.8.252@KRBTEST.COM (short username kudu)
I20260926 16:36:43.891669 1271 webserver.cc:460] Webserver started at http://127.11.8.252:43327/ using document root <none> and password file <none>
I20260926 16:36:43.892201 1271 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:43.892261 1271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:43.892490 1271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:43.894543 1271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-2/data/instance:
uuid: "e38abc45a6034fe2bae74498dcca31e5"
format_stamp: "Formatted at 2026-09-26 16:36:43 on dist-test-slave-2zvq"
server_key: "ea691f6db20eeb04e298bcf5a846371f"
server_key_iv: "288f67b279bc82e852b63cf1c07f0ef0"
server_key_version: "encryptionkey@0"
I20260926 16:36:43.895108 1271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-2/wal/instance:
uuid: "e38abc45a6034fe2bae74498dcca31e5"
format_stamp: "Formatted at 2026-09-26 16:36:43 on dist-test-slave-2zvq"
server_key: "ea691f6db20eeb04e298bcf5a846371f"
server_key_iv: "288f67b279bc82e852b63cf1c07f0ef0"
server_key_version: "encryptionkey@0"
I20260926 16:36:43.898813 1271 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 16:36:43.901162 1286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:43.902516 1271 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:36:43.902647 1271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-2/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-2/wal
uuid: "e38abc45a6034fe2bae74498dcca31e5"
format_stamp: "Formatted at 2026-09-26 16:36:43 on dist-test-slave-2zvq"
server_key: "ea691f6db20eeb04e298bcf5a846371f"
server_key_iv: "288f67b279bc82e852b63cf1c07f0ef0"
server_key_version: "encryptionkey@0"
I20260926 16:36:43.902771 1271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:43.918519 1271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:43.921747 1271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:43.921947 1271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:43.928692 1271 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.252:36607
I20260926 16:36:43.928966 1339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.252:36607 every 8 connection(s)
I20260926 16:36:43.929673 1271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-2/data/info.pb
I20260926 16:36:43.932435 1340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:43.935999 1340 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.254" port: 44201 } has no permanent_uuid. Determining permanent_uuid...
I20260926 16:36:43.939728 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 1271
I20260926 16:36:43.939875 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790440324823152-11299-0/minicluster-data/master-2/wal/instance
I20260926 16:36:43.940146 11299 external_mini_cluster.cc:1428] Setting key c04335479824c12ec8b296df826c1d35
Sep 26 16:36:43 dist-test-slave-2zvq krb5kdc[1106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440603, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.252@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
Sep 26 16:36:43 dist-test-slave-2zvq krb5kdc[1106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.252@KRBTEST.COM
I20260926 16:36:43.950093 1191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:43.937795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36514 (local address 127.11.8.254:44201)
0926 16:36:43.937953 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:43.937957 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:43.938739 (+ 782us) server_negotiation.cc:407] Connection header received
0926 16:36:43.939697 (+ 958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:43.939700 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:43.939759 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:43.939847 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:43.941469 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:43.941968 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:43.943148 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:43.943367 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:43.943485 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:43.946246 (+ 2761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:43.946255 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:43.946257 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:43.946320 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:43.947793 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:43.948426 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:43.948433 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:43.948436 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:43.948523 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:43.948910 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:43.948913 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:43.948914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:43.949105 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:43.949193 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:43.949721 (+ 528us) server_negotiation.cc:299] Negotiation successful
0926 16:36:43.949898 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 16:36:43.950992 1342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:43.938100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:44201 (local address 127.0.0.1:36514)
0926 16:36:43.938593 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:43.938633 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:43.939484 (+ 851us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:43.940087 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:43.940097 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:43.940558 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:36:43.941264 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:43.941284 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:43.942092 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:43.942096 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:43.943023 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:43.943032 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:43.943514 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:43.943518 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:43.943693 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:43.944315 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:36:43.944333 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:36:43.946104 (+ 1771us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:36:43.947901 (+ 1797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:43.947911 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:43.947921 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:43.948301 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:43.948617 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:43.948620 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:43.948622 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:43.948787 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:43.949217 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:36:43.949226 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:36:43.949477 (+ 251us) client_negotiation.cc:769] Sending connection context
0926 16:36:43.949668 (+ 191us) client_negotiation.cc:240] Negotiation successful
0926 16:36:43.949957 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":166,"threads_started":1}
I20260926 16:36:43.950992 1344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:43.940824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41238 (local address 127.11.8.252:36607)
0926 16:36:43.941233 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:43.941246 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:43.941275 (+ 29us) server_negotiation.cc:407] Connection header received
0926 16:36:43.941375 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:43.941380 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:43.941558 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:43.941728 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:43.942536 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:43.943254 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:43.944190 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:43.944734 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:43.944852 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:43.947344 (+ 2492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:43.947349 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:43.947357 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:43.947377 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:43.949186 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:43.949619 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:43.949627 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:43.949632 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:43.949712 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:43.950024 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:43.950027 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:43.950028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:43.950301 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:43.950505 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:43.950680 (+ 175us) server_negotiation.cc:299] Negotiation successful
0926 16:36:43.950806 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":159,"threads_started":1}
I20260926 16:36:43.952172 1340 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.253" port: 41685 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 16:36:43 dist-test-slave-2zvq krb5kdc[1106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440603, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.252@KRBTEST.COM for kudu/127.11.8.253@KRBTEST.COM
Sep 26 16:36:43 dist-test-slave-2zvq krb5kdc[1106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:36:43.961123 1269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:43.952801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57376 (local address 127.11.8.253:41685)
0926 16:36:43.952935 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:43.952939 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:43.952950 (+ 11us) server_negotiation.cc:407] Connection header received
0926 16:36:43.952985 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:43.952987 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:43.953027 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:43.953160 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:43.953630 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:43.954501 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:43.955478 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:43.955853 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:43.955954 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:43.957978 (+ 2024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:43.957981 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:43.957983 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:43.958004 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:43.959491 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:43.959947 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:43.959949 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:43.959950 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:43.960023 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:43.960337 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:43.960343 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:43.960347 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:43.960526 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:43.960605 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:43.960881 (+ 276us) server_negotiation.cc:299] Negotiation successful
0926 16:36:43.960989 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 16:36:43.961133 1342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:43.952628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.253:41685 (local address 127.0.0.1:57376)
0926 16:36:43.952731 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:43.952749 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:43.952813 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:43.953179 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:43.953181 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:43.953366 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:36:43.953498 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:43.953505 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:43.954592 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:43.954595 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:43.955366 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:43.955374 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:43.955954 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:43.955957 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:43.956025 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:43.956490 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:36:43.956504 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:36:43.957808 (+ 1304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:36:43.959591 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:43.959594 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:43.959596 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:43.959838 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:43.960117 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:43.960122 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:43.960127 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:43.960225 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:43.960624 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:36:43.960630 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:36:43.960719 (+ 89us) client_negotiation.cc:769] Sending connection context
0926 16:36:43.960837 (+ 118us) client_negotiation.cc:240] Negotiation successful
0926 16:36:43.960970 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260926 16:36:43.961897 1340 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.252" port: 36607 } has no permanent_uuid. Determining permanent_uuid...
I20260926 16:36:43.962785 1191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:43.953384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36516 (local address 127.11.8.254:44201)
0926 16:36:43.953512 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:43.953516 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:43.953531 (+ 15us) server_negotiation.cc:407] Connection header received
0926 16:36:43.953622 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:43.953624 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:43.953663 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:43.953725 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:43.954442 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:43.955154 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:43.956145 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:43.956406 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:43.956510 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:43.959437 (+ 2927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:43.959440 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:43.959443 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:43.959466 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:43.961041 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:43.961525 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:43.961529 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:43.961530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:43.961618 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:43.961944 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:43.961949 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:43.961951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:43.962159 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:43.962409 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:43.962530 (+ 121us) server_negotiation.cc:299] Negotiation successful
0926 16:36:43.962635 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
Sep 26 16:36:43 dist-test-slave-2zvq krb5kdc[1106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440603, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.252@KRBTEST.COM for kudu/127.11.8.252@KRBTEST.COM
I20260926 16:36:43.966888 1263 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.252" port: 36607 } attempt: 3
I20260926 16:36:43.968691 1188 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.252" port: 36607 } attempt: 3
Sep 26 16:36:43 dist-test-slave-2zvq krb5kdc[1106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.253@KRBTEST.COM
I20260926 16:36:43.970502 1342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:43.962395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:36607 (local address 127.0.0.1:41248)
0926 16:36:43.962475 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:43.962487 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:43.962556 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:43.962839 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:43.962844 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:43.963006 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:36:43.963099 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:43.963108 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:43.963948 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:43.963952 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:43.964501 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:43.964510 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:43.965105 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:43.965107 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:43.965184 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:43.965586 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:36:43.965596 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:36:43.966903 (+ 1307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:36:43.969012 (+ 2109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:43.969018 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:43.969022 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:43.969245 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:43.969545 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:43.969548 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:43.969549 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:43.969633 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:43.970018 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:36:43.970021 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:36:43.970097 (+ 76us) client_negotiation.cc:769] Sending connection context
0926 16:36:43.970176 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 16:36:43.970306 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24,"mutex_wait_us":3}
I20260926 16:36:43.971299 1344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:43.962554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41248 (local address 127.11.8.252:36607)
0926 16:36:43.962658 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:43.962661 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:43.962672 (+ 11us) server_negotiation.cc:407] Connection header received
0926 16:36:43.962700 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:43.962702 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:43.962742 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:43.962814 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:43.963217 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:43.963863 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:43.964594 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:43.964997 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:43.965152 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:43.967032 (+ 1880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:43.967035 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:43.967038 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:43.967061 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:43.968901 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:43.969356 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:43.969362 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:43.969367 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:43.969456 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:43.969735 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:43.969738 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:43.969742 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:43.969925 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:43.970983 (+ 1058us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:43.971066 (+ 83us) server_negotiation.cc:299] Negotiation successful
0926 16:36:43.971168 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":39}
Sep 26 16:36:43 dist-test-slave-2zvq krb5kdc[1106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440603, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.253@KRBTEST.COM for kudu/127.11.8.252@KRBTEST.COM
Sep 26 16:36:43 dist-test-slave-2zvq krb5kdc[1106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440603, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for kudu/127.11.8.252@KRBTEST.COM
I20260926 16:36:43.974843 1269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:43.964080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57388 (local address 127.11.8.253:41685)
0926 16:36:43.964188 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:43.964192 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:43.964204 (+ 12us) server_negotiation.cc:407] Connection header received
0926 16:36:43.964331 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:43.964334 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:43.964371 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:43.964458 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:43.965133 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:43.965804 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:43.966896 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:43.967225 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:43.967636 (+ 411us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:43.970514 (+ 2878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:43.970517 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:43.970520 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:43.970542 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:43.972650 (+ 2108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:43.973095 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:43.973099 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:43.973100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:43.973182 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:43.973508 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:43.973511 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:43.973513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:43.973721 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:43.973928 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:43.974581 (+ 653us) server_negotiation.cc:299] Negotiation successful
0926 16:36:43.974684 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 16:36:43.975554 1340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e38abc45a6034fe2bae74498dcca31e5: Bootstrap starting.
I20260926 16:36:43.978396 11299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:36:43.978768 1265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:43.967269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:36607 (local address 127.0.0.1:41262)
0926 16:36:43.967467 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:43.967483 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:43.967555 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:43.968028 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:43.968031 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:43.968190 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:36:43.968255 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:43.968259 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:43.969892 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:43.969896 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:43.971022 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:43.971034 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:43.971710 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:43.971714 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:43.971808 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:43.972212 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:36:43.972229 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:36:43.974019 (+ 1790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:36:43.976856 (+ 2837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:43.976861 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:43.976864 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:43.977117 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:43.977766 (+ 649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:43.977769 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:43.977771 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:43.977862 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:43.978308 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:36:43.978311 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:36:43.978397 (+ 86us) client_negotiation.cc:769] Sending connection context
0926 16:36:43.978476 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 16:36:43.978609 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91}
I20260926 16:36:43.978946 1340 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e38abc45a6034fe2bae74498dcca31e5: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:43.978981 1192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:43.969096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:36607 (local address 127.0.0.1:41264)
0926 16:36:43.969326 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:43.969341 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:43.969402 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:43.970033 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:43.970036 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:43.970210 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:36:43.970328 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:43.970335 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:43.971454 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:43.971457 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:43.972161 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:43.972170 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:43.973079 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:43.973083 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:43.973169 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:43.973586 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:36:43.973601 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:36:43.975015 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:36:43.977223 (+ 2208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:43.977226 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:43.977228 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:43.977486 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:43.978023 (+ 537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:43.978026 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:43.978028 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:43.978107 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:43.978537 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:36:43.978540 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:36:43.978631 (+ 91us) client_negotiation.cc:769] Sending connection context
0926 16:36:43.978710 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 16:36:43.978820 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151}
I20260926 16:36:43.979305 1345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:43.967304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41262 (local address 127.11.8.252:36607)
0926 16:36:43.967795 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:43.967807 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:43.967822 (+ 15us) server_negotiation.cc:407] Connection header received
0926 16:36:43.967857 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:43.967863 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:43.967907 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:43.968013 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:43.968379 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:43.969749 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:43.971173 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:43.971573 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:43.972013 (+ 440us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:43.974166 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:43.974169 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:43.974171 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:43.974194 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:43.976728 (+ 2534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:43.977230 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:43.977234 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:43.977235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:43.977652 (+ 417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:43.977965 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:43.977969 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:43.977972 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:43.978169 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:43.978929 (+ 760us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:43.979003 (+ 74us) server_negotiation.cc:299] Negotiation successful
0926 16:36:43.979116 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":113,"threads_started":1}
I20260926 16:36:43.979489 1346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:43.969538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41264 (local address 127.11.8.252:36607)
0926 16:36:43.969805 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:43.969814 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:43.969825 (+ 11us) server_negotiation.cc:407] Connection header received
0926 16:36:43.969852 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:43.969855 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:43.969889 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:43.970430 (+ 541us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:43.970542 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:43.971317 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:43.972587 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:43.972938 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:43.973367 (+ 429us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:43.975163 (+ 1796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:43.975166 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:43.975169 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:43.975205 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:43.977082 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:43.977821 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:43.977823 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:43.977825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:43.977891 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:43.978219 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:43.978223 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:43.978224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:43.978422 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:43.979125 (+ 703us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:43.979189 (+ 64us) server_negotiation.cc:299] Negotiation successful
0926 16:36:43.979296 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":111,"threads_started":1}
I20260926 16:36:43.979900 1340 log.cc:824] T 00000000000000000000000000000000 P e38abc45a6034fe2bae74498dcca31e5: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:43.980580 1161 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "faca929a73f642f1b760ff7467020f53"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36516
I20260926 16:36:43.982642 1188 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faca929a73f642f1b760ff7467020f53: Bootstrap starting.
I20260926 16:36:43.982679 1263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1737b62f0bc74b0a97782c30264530aa: Bootstrap starting.
I20260926 16:36:43.984611 1188 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P faca929a73f642f1b760ff7467020f53: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:43.984963 1263 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1737b62f0bc74b0a97782c30264530aa: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:43.985442 1188 log.cc:824] T 00000000000000000000000000000000 P faca929a73f642f1b760ff7467020f53: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:43.985603 1263 log.cc:824] T 00000000000000000000000000000000 P 1737b62f0bc74b0a97782c30264530aa: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:43.987427 1188 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faca929a73f642f1b760ff7467020f53: No bootstrap required, opened a new log
I20260926 16:36:43.987470 1263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1737b62f0bc74b0a97782c30264530aa: No bootstrap required, opened a new log
I20260926 16:36:43.987478 1340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e38abc45a6034fe2bae74498dcca31e5: No bootstrap required, opened a new log
I20260926 16:36:43.990379 1188 raft_consensus.cc:348] T 00000000000000000000000000000000 P faca929a73f642f1b760ff7467020f53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faca929a73f642f1b760ff7467020f53" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 44201 } } peers { permanent_uuid: "1737b62f0bc74b0a97782c30264530aa" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 41685 } } peers { permanent_uuid: "e38abc45a6034fe2bae74498dcca31e5" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 36607 } }
I20260926 16:36:43.990590 1188 raft_consensus.cc:729] T 00000000000000000000000000000000 P faca929a73f642f1b760ff7467020f53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: faca929a73f642f1b760ff7467020f53, State: Initialized, Role: FOLLOWER
I20260926 16:36:43.990917 1188 consensus_queue.cc:260] T 00000000000000000000000000000000 P faca929a73f642f1b760ff7467020f53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faca929a73f642f1b760ff7467020f53" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 44201 } } peers { permanent_uuid: "1737b62f0bc74b0a97782c30264530aa" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 41685 } } peers { permanent_uuid: "e38abc45a6034fe2bae74498dcca31e5" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 36607 } }
I20260926 16:36:43.991333 1263 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1737b62f0bc74b0a97782c30264530aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faca929a73f642f1b760ff7467020f53" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 44201 } } peers { permanent_uuid: "1737b62f0bc74b0a97782c30264530aa" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 41685 } } peers { permanent_uuid: "e38abc45a6034fe2bae74498dcca31e5" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 36607 } }
I20260926 16:36:43.991338 1340 raft_consensus.cc:348] T 00000000000000000000000000000000 P e38abc45a6034fe2bae74498dcca31e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faca929a73f642f1b760ff7467020f53" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 44201 } } peers { permanent_uuid: "1737b62f0bc74b0a97782c30264530aa" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 41685 } } peers { permanent_uuid: "e38abc45a6034fe2bae74498dcca31e5" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 36607 } }
I20260926 16:36:43.991546 1263 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1737b62f0bc74b0a97782c30264530aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1737b62f0bc74b0a97782c30264530aa, State: Initialized, Role: FOLLOWER
I20260926 16:36:43.991554 1340 raft_consensus.cc:729] T 00000000000000000000000000000000 P e38abc45a6034fe2bae74498dcca31e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e38abc45a6034fe2bae74498dcca31e5, State: Initialized, Role: FOLLOWER
I20260926 16:36:43.991870 1340 consensus_queue.cc:260] T 00000000000000000000000000000000 P e38abc45a6034fe2bae74498dcca31e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faca929a73f642f1b760ff7467020f53" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 44201 } } peers { permanent_uuid: "1737b62f0bc74b0a97782c30264530aa" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 41685 } } peers { permanent_uuid: "e38abc45a6034fe2bae74498dcca31e5" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 36607 } }
I20260926 16:36:43.991864 1263 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1737b62f0bc74b0a97782c30264530aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faca929a73f642f1b760ff7467020f53" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 44201 } } peers { permanent_uuid: "1737b62f0bc74b0a97782c30264530aa" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 41685 } } peers { permanent_uuid: "e38abc45a6034fe2bae74498dcca31e5" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 36607 } }
I20260926 16:36:43.992220 1188 sys_catalog.cc:564] T 00000000000000000000000000000000 P faca929a73f642f1b760ff7467020f53 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:36:43.992292 1161 raft_consensus.cc:482] T 00000000000000000000000000000000 P faca929a73f642f1b760ff7467020f53 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260926 16:36:43.992383 1161 raft_consensus.cc:3037] T 00000000000000000000000000000000 P faca929a73f642f1b760ff7467020f53 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:43.993084 1340 sys_catalog.cc:564] T 00000000000000000000000000000000 P e38abc45a6034fe2bae74498dcca31e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:36:43.993690 1161 raft_consensus.cc:504] T 00000000000000000000000000000000 P faca929a73f642f1b760ff7467020f53 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faca929a73f642f1b760ff7467020f53" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 44201 } } peers { permanent_uuid: "1737b62f0bc74b0a97782c30264530aa" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 41685 } } peers { permanent_uuid: "e38abc45a6034fe2bae74498dcca31e5" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 36607 } }
I20260926 16:36:43.995955 1352 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1737b62f0bc74b0a97782c30264530aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faca929a73f642f1b760ff7467020f53" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 44201 } } peers { permanent_uuid: "1737b62f0bc74b0a97782c30264530aa" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 41685 } } peers { permanent_uuid: "e38abc45a6034fe2bae74498dcca31e5" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 36607 } } }
I20260926 16:36:43.996181 1352 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1737b62f0bc74b0a97782c30264530aa [sys.catalog]: This master's current role is: FOLLOWER
W20260926 16:36:44.001385 1140 tablet.cc:2369] T 00000000000000000000000000000000 P faca929a73f642f1b760ff7467020f53: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:36:43.997993 1119 master_runner.cc:186] Error getting master registration for 127.11.8.253:41685: OK, instance_id { permanent_uuid: "1737b62f0bc74b0a97782c30264530aa" instance_seqno: 1790440603711732 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 16:36:44.001637 1119 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 16:36:43.994707 1161 leader_election.cc:288] T 00000000000000000000000000000000 P faca929a73f642f1b760ff7467020f53 [CANDIDATE]: Term 1 election: Requested vote from peers 1737b62f0bc74b0a97782c30264530aa (127.11.8.253:41685), e38abc45a6034fe2bae74498dcca31e5 (127.11.8.252:36607)
I20260926 16:36:44.002239 1350 sys_catalog.cc:455] T 00000000000000000000000000000000 P faca929a73f642f1b760ff7467020f53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faca929a73f642f1b760ff7467020f53" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 44201 } } peers { permanent_uuid: "1737b62f0bc74b0a97782c30264530aa" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 41685 } } peers { permanent_uuid: "e38abc45a6034fe2bae74498dcca31e5" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 36607 } } }
I20260926 16:36:44.002390 1350 sys_catalog.cc:458] T 00000000000000000000000000000000 P faca929a73f642f1b760ff7467020f53 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:36:43.995450 1313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "faca929a73f642f1b760ff7467020f53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e38abc45a6034fe2bae74498dcca31e5"
I20260926 16:36:44.002769 1313 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e38abc45a6034fe2bae74498dcca31e5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:44.004228 1313 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e38abc45a6034fe2bae74498dcca31e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate faca929a73f642f1b760ff7467020f53 in term 1.
I20260926 16:36:43.995713 1351 sys_catalog.cc:455] T 00000000000000000000000000000000 P e38abc45a6034fe2bae74498dcca31e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faca929a73f642f1b760ff7467020f53" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 44201 } } peers { permanent_uuid: "1737b62f0bc74b0a97782c30264530aa" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 41685 } } peers { permanent_uuid: "e38abc45a6034fe2bae74498dcca31e5" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 36607 } } }
W20260926 16:36:43.998872 1374 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e38abc45a6034fe2bae74498dcca31e5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260926 16:36:44.004467 1351 sys_catalog.cc:458] T 00000000000000000000000000000000 P e38abc45a6034fe2bae74498dcca31e5 [sys.catalog]: This master's current role is: FOLLOWER
W20260926 16:36:44.004485 1374 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 16:36:44.004705 1135 leader_election.cc:302] T 00000000000000000000000000000000 P faca929a73f642f1b760ff7467020f53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e38abc45a6034fe2bae74498dcca31e5, faca929a73f642f1b760ff7467020f53; no voters:
I20260926 16:36:44.005184 1263 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1737b62f0bc74b0a97782c30264530aa [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:36:44.005471 1237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "faca929a73f642f1b760ff7467020f53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1737b62f0bc74b0a97782c30264530aa"
I20260926 16:36:44.005607 1350 raft_consensus.cc:2781] T 00000000000000000000000000000000 P faca929a73f642f1b760ff7467020f53 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:36:44.005700 1237 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1737b62f0bc74b0a97782c30264530aa [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:44.007078 1237 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1737b62f0bc74b0a97782c30264530aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate faca929a73f642f1b760ff7467020f53 in term 1.
I20260926 16:36:44.006232 1271 master_runner.cc:186] Error getting master registration for 127.11.8.253:41685: OK, instance_id { permanent_uuid: "1737b62f0bc74b0a97782c30264530aa" instance_seqno: 1790440603711732 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 16:36:44.009634 1271 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 16:36:44.009800 1350 raft_consensus.cc:686] T 00000000000000000000000000000000 P faca929a73f642f1b760ff7467020f53 [term 1 LEADER]: Becoming Leader. State: Replica: faca929a73f642f1b760ff7467020f53, State: Running, Role: LEADER
I20260926 16:36:44.010164 1350 consensus_queue.cc:237] T 00000000000000000000000000000000 P faca929a73f642f1b760ff7467020f53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faca929a73f642f1b760ff7467020f53" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 44201 } } peers { permanent_uuid: "1737b62f0bc74b0a97782c30264530aa" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 41685 } } peers { permanent_uuid: "e38abc45a6034fe2bae74498dcca31e5" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 36607 } }
I20260926 16:36:44.011963 1350 sys_catalog.cc:455] T 00000000000000000000000000000000 P faca929a73f642f1b760ff7467020f53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader faca929a73f642f1b760ff7467020f53. Latest consensus state: current_term: 1 leader_uuid: "faca929a73f642f1b760ff7467020f53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faca929a73f642f1b760ff7467020f53" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 44201 } } peers { permanent_uuid: "1737b62f0bc74b0a97782c30264530aa" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 41685 } } peers { permanent_uuid: "e38abc45a6034fe2bae74498dcca31e5" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 36607 } } }
I20260926 16:36:44.012069 1350 sys_catalog.cc:458] T 00000000000000000000000000000000 P faca929a73f642f1b760ff7467020f53 [sys.catalog]: This master's current role is: LEADER
W20260926 16:36:44.018666 1365 catalog_manager.cc:1558] T 00000000000000000000000000000000 P faca929a73f642f1b760ff7467020f53: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 16:36:44.018739 1365 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 16:36:44.022764 1269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:44.012532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57390 (local address 127.11.8.253:41685)
0926 16:36:44.012638 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:44.012643 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:44.013582 (+ 939us) server_negotiation.cc:407] Connection header received
0926 16:36:44.013758 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:44.013768 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:44.013813 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:44.013869 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:44.014771 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:44.015513 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:44.016655 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:44.016988 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:44.017060 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:44.018985 (+ 1925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:44.018988 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:44.018991 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:44.019015 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:44.020864 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:44.021355 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:44.021359 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:44.021361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:44.021444 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:44.021759 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:44.021762 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:44.021764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:44.021981 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:44.022083 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:44.022490 (+ 407us) server_negotiation.cc:299] Negotiation successful
0926 16:36:44.022607 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260926 16:36:44.023665 1392 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1737b62f0bc74b0a97782c30264530aa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 16:36:44.023746 1392 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 16:36:44.026363 1394 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:36:44.027302 1394 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:36:44.029428 1191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:44.004053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36530 (local address 127.11.8.254:44201)
0926 16:36:44.004164 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:44.004169 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:44.009705 (+ 5536us) server_negotiation.cc:407] Connection header received
0926 16:36:44.011042 (+ 1337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:44.011046 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:44.011096 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:44.011149 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:44.017890 (+ 6741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:44.018663 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:44.019823 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:44.020162 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:44.020233 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:44.025706 (+ 5473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:44.025710 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:44.025713 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:44.025739 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:44.027527 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:44.028045 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:44.028049 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:44.028051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:44.028133 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:44.028446 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:44.028451 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:44.028453 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:44.028662 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:44.028786 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:44.029150 (+ 364us) server_negotiation.cc:299] Negotiation successful
0926 16:36:44.029272 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 16:36:44.038394 1346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:44.013217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41266 (local address 127.11.8.252:36607)
0926 16:36:44.013322 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:44.013327 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:44.024249 (+ 10922us) server_negotiation.cc:407] Connection header received
0926 16:36:44.024421 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:44.024424 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:44.024473 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:44.024523 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:44.030297 (+ 5774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:44.031096 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:44.032229 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:44.032603 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:44.032711 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:44.034239 (+ 1528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:44.034243 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:44.034246 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:44.034302 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:44.036668 (+ 2366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:44.037149 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:44.037152 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:44.037154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:44.037243 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:44.037528 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:44.037531 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:44.037532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:44.037734 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:44.037873 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:44.038103 (+ 230us) server_negotiation.cc:299] Negotiation successful
0926 16:36:44.038217 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 16:36:44.046829 1313 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e38abc45a6034fe2bae74498dcca31e5 [term 1 FOLLOWER]: Refusing update from remote peer faca929a73f642f1b760ff7467020f53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:36:44.047042 1237 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1737b62f0bc74b0a97782c30264530aa [term 1 FOLLOWER]: Refusing update from remote peer faca929a73f642f1b760ff7467020f53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:36:44.047502 1381 consensus_queue.cc:1035] T 00000000000000000000000000000000 P faca929a73f642f1b760ff7467020f53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e38abc45a6034fe2bae74498dcca31e5" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 36607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:36:44.052109 1351 sys_catalog.cc:455] T 00000000000000000000000000000000 P e38abc45a6034fe2bae74498dcca31e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader faca929a73f642f1b760ff7467020f53. Latest consensus state: current_term: 1 leader_uuid: "faca929a73f642f1b760ff7467020f53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faca929a73f642f1b760ff7467020f53" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 44201 } } peers { permanent_uuid: "1737b62f0bc74b0a97782c30264530aa" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 41685 } } peers { permanent_uuid: "e38abc45a6034fe2bae74498dcca31e5" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 36607 } } }
I20260926 16:36:44.052225 1351 sys_catalog.cc:458] T 00000000000000000000000000000000 P e38abc45a6034fe2bae74498dcca31e5 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:36:44.052591 1381 consensus_queue.cc:1035] T 00000000000000000000000000000000 P faca929a73f642f1b760ff7467020f53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1737b62f0bc74b0a97782c30264530aa" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 41685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:36:44.056547 1352 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1737b62f0bc74b0a97782c30264530aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader faca929a73f642f1b760ff7467020f53. Latest consensus state: current_term: 1 leader_uuid: "faca929a73f642f1b760ff7467020f53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faca929a73f642f1b760ff7467020f53" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 44201 } } peers { permanent_uuid: "1737b62f0bc74b0a97782c30264530aa" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 41685 } } peers { permanent_uuid: "e38abc45a6034fe2bae74498dcca31e5" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 36607 } } }
I20260926 16:36:44.056677 1352 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1737b62f0bc74b0a97782c30264530aa [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:36:44.062120 1381 sys_catalog.cc:455] T 00000000000000000000000000000000 P faca929a73f642f1b760ff7467020f53 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "faca929a73f642f1b760ff7467020f53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faca929a73f642f1b760ff7467020f53" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 44201 } } peers { permanent_uuid: "1737b62f0bc74b0a97782c30264530aa" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 41685 } } peers { permanent_uuid: "e38abc45a6034fe2bae74498dcca31e5" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 36607 } } }
I20260926 16:36:44.062253 1381 sys_catalog.cc:458] T 00000000000000000000000000000000 P faca929a73f642f1b760ff7467020f53 [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:44.063289 1351 sys_catalog.cc:455] T 00000000000000000000000000000000 P e38abc45a6034fe2bae74498dcca31e5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "faca929a73f642f1b760ff7467020f53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faca929a73f642f1b760ff7467020f53" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 44201 } } peers { permanent_uuid: "1737b62f0bc74b0a97782c30264530aa" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 41685 } } peers { permanent_uuid: "e38abc45a6034fe2bae74498dcca31e5" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 36607 } } }
I20260926 16:36:44.063414 1351 sys_catalog.cc:458] T 00000000000000000000000000000000 P e38abc45a6034fe2bae74498dcca31e5 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:36:44.063755 1399 mvcc.cc:205] Tried to move back new op lower bound from 7333644714167971840 to 7333644714029133824. Current Snapshot: MvccSnapshot[applied={T|T < 7333644714167971840}]
I20260926 16:36:44.066968 1350 sys_catalog.cc:455] T 00000000000000000000000000000000 P faca929a73f642f1b760ff7467020f53 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "faca929a73f642f1b760ff7467020f53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faca929a73f642f1b760ff7467020f53" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 44201 } } peers { permanent_uuid: "1737b62f0bc74b0a97782c30264530aa" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 41685 } } peers { permanent_uuid: "e38abc45a6034fe2bae74498dcca31e5" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 36607 } } }
I20260926 16:36:44.067082 1350 sys_catalog.cc:458] T 00000000000000000000000000000000 P faca929a73f642f1b760ff7467020f53 [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:44.067188 1394 catalog_manager.cc:1347] Generated new cluster ID: 5afaf97e33d74a1baec35766b6307391
I20260926 16:36:44.067255 1394 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:36:44.068323 1352 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1737b62f0bc74b0a97782c30264530aa [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "faca929a73f642f1b760ff7467020f53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faca929a73f642f1b760ff7467020f53" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 44201 } } peers { permanent_uuid: "1737b62f0bc74b0a97782c30264530aa" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 41685 } } peers { permanent_uuid: "e38abc45a6034fe2bae74498dcca31e5" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 36607 } } }
I20260926 16:36:44.068444 1352 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1737b62f0bc74b0a97782c30264530aa [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:36:44.069434 1400 mvcc.cc:205] Tried to move back new op lower bound from 7333644714167971840 to 7333644714029133824. Current Snapshot: MvccSnapshot[applied={T|T < 7333644714167971840}]
I20260926 16:36:44.074349 1402 mvcc.cc:205] Tried to move back new op lower bound from 7333644714167971840 to 7333644714029133824. Current Snapshot: MvccSnapshot[applied={T|T < 7333644714167971840 or (T in {7333644714167971840})}]
I20260926 16:36:44.087743 1394 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:36:44.089030 1394 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:36:44.099648 1394 catalog_manager.cc:5868] T 00000000000000000000000000000000 P faca929a73f642f1b760ff7467020f53: Generated new TSK 0
I20260926 16:36:44.100446 1394 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 16:36:44.152717 1269 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:36:44.149578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57394 (local address 127.11.8.253:41685)
0926 16:36:44.149704 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:44.149709 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:44.151672 (+ 1963us) server_negotiation.cc:407] Connection header received
0926 16:36:44.151864 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:44.151868 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:44.151926 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:44.151979 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:44.152577 (+ 598us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57394: BlockingRecv error: recv got EOF from 127.0.0.1:57394 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260926 16:36:44.154790 1191 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:36:44.150106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36536 (local address 127.11.8.254:44201)
0926 16:36:44.150226 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:44.150230 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:44.153233 (+ 3003us) server_negotiation.cc:407] Connection header received
0926 16:36:44.153786 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:44.153789 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:44.153838 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:44.153900 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:44.154669 (+ 769us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36536: BlockingRecv error: recv got EOF from 127.0.0.1:36536 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260926 16:36:44.155616 1346 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:36:44.150826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41274 (local address 127.11.8.252:36607)
0926 16:36:44.150947 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:44.150951 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:44.153541 (+ 2590us) server_negotiation.cc:407] Connection header received
0926 16:36:44.154028 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:44.154031 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:44.154076 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:44.154125 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:44.155508 (+ 1383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41274: BlockingRecv error: recv got EOF from 127.0.0.1:41274 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260926 16:36:44.159970 1269 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:36:44.158462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57396 (local address 127.11.8.253:41685)
0926 16:36:44.158554 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:44.158558 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:44.159059 (+ 501us) server_negotiation.cc:407] Connection header received
0926 16:36:44.159270 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:44.159273 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:44.159341 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:44.159392 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:44.159847 (+ 455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57396: BlockingRecv error: recv got EOF from 127.0.0.1:57396 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260926 16:36:44.164938 1269 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:36:44.163469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57406 (local address 127.11.8.253:41685)
0926 16:36:44.163557 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:44.163561 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:44.164120 (+ 559us) server_negotiation.cc:407] Connection header received
0926 16:36:44.164293 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:44.164296 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:44.164341 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:44.164390 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:44.164838 (+ 448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57406: BlockingRecv error: recv got EOF from 127.0.0.1:57406 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260926 16:36:44.170415 1269 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:36:44.168914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57414 (local address 127.11.8.253:41685)
0926 16:36:44.169006 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:44.169020 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:44.169564 (+ 544us) server_negotiation.cc:407] Connection header received
0926 16:36:44.169735 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:44.169738 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:44.169786 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:44.169836 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:44.170332 (+ 496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57414: BlockingRecv error: recv got EOF from 127.0.0.1:57414 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260926 16:36:44.178229 1269 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:36:44.176503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57418 (local address 127.11.8.253:41685)
0926 16:36:44.176620 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:44.176624 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:44.177236 (+ 612us) server_negotiation.cc:407] Connection header received
0926 16:36:44.177525 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:44.177531 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:44.177589 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:44.177642 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:44.178118 (+ 476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57418: BlockingRecv error: recv got EOF from 127.0.0.1:57418 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260926 16:36:44.191021 1269 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:36:44.189168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57428 (local address 127.11.8.253:41685)
0926 16:36:44.189332 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:44.189337 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:44.189930 (+ 593us) server_negotiation.cc:407] Connection header received
0926 16:36:44.190165 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:44.190170 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:44.190225 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:44.190318 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:44.190906 (+ 588us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57428: BlockingRecv error: recv got EOF from 127.0.0.1:57428 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
W20260926 16:36:44.215682 1269 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:36:44.210715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57432 (local address 127.11.8.253:41685)
0926 16:36:44.210834 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:44.210838 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:44.214600 (+ 3762us) server_negotiation.cc:407] Connection header received
0926 16:36:44.214832 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:44.214836 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:44.214900 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:44.214954 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:44.215545 (+ 591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57432: BlockingRecv error: recv got EOF from 127.0.0.1:57432 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260926 16:36:44.251780 1269 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:36:44.250494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57440 (local address 127.11.8.253:41685)
0926 16:36:44.250642 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:44.250647 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:44.250856 (+ 209us) server_negotiation.cc:407] Connection header received
0926 16:36:44.251012 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:44.251015 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:44.251061 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:44.251135 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:44.251646 (+ 511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57440: BlockingRecv error: recv got EOF from 127.0.0.1:57440 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260926 16:36:44.319743 1269 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:36:44.318492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57448 (local address 127.11.8.253:41685)
0926 16:36:44.318681 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:44.318686 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:44.318840 (+ 154us) server_negotiation.cc:407] Connection header received
0926 16:36:44.319040 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:44.319044 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:44.319089 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:44.319178 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:44.319642 (+ 464us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57448: BlockingRecv error: recv got EOF from 127.0.0.1:57448 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260926 16:36:44.452505 1269 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:36:44.451253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57458 (local address 127.11.8.253:41685)
0926 16:36:44.451412 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:44.451417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:44.451610 (+ 193us) server_negotiation.cc:407] Connection header received
0926 16:36:44.451697 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:44.451700 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:44.451749 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:44.451830 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:44.452348 (+ 518us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57458: BlockingRecv error: recv got EOF from 127.0.0.1:57458 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260926 16:36:44.711949 1269 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:36:44.710708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57462 (local address 127.11.8.253:41685)
0926 16:36:44.710864 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:44.710868 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:44.711002 (+ 134us) server_negotiation.cc:407] Connection header received
0926 16:36:44.711177 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:44.711180 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:44.711234 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:44.711360 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:44.711834 (+ 474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57462: BlockingRecv error: recv got EOF from 127.0.0.1:57462 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 16:36:45.006160 1374 catalog_manager.cc:1259] Loaded cluster ID: 5afaf97e33d74a1baec35766b6307391
I20260926 16:36:45.006314 1374 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e38abc45a6034fe2bae74498dcca31e5: loading cluster ID for follower catalog manager: success
I20260926 16:36:45.008212 1374 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e38abc45a6034fe2bae74498dcca31e5: acquiring CA information for follower catalog manager: success
I20260926 16:36:45.009382 1374 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e38abc45a6034fe2bae74498dcca31e5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 16:36:45.025152 1392 catalog_manager.cc:1259] Loaded cluster ID: 5afaf97e33d74a1baec35766b6307391
I20260926 16:36:45.025238 1392 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1737b62f0bc74b0a97782c30264530aa: loading cluster ID for follower catalog manager: success
I20260926 16:36:45.027446 1392 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1737b62f0bc74b0a97782c30264530aa: acquiring CA information for follower catalog manager: success
I20260926 16:36:45.028843 1392 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1737b62f0bc74b0a97782c30264530aa: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 16:36:45.230515 1473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:45.226240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57478 (local address 127.11.8.253:41685)
0926 16:36:45.226569 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:45.226574 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:45.226735 (+ 161us) server_negotiation.cc:407] Connection header received
0926 16:36:45.226900 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:45.226904 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:45.227041 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:45.227166 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:36:45.227461 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:45.228011 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:45.228923 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:45.229152 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:45.229268 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:45.229606 (+ 338us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:36:45.230057 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:36:45.230204 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:45.230294 (+ 90us) server_negotiation.cc:299] Negotiation successful
0926 16:36:45.230354 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":119,"threads_started":1}
I20260926 16:36:45.237334 1479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:45.233683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41276 (local address 127.11.8.252:36607)
0926 16:36:45.233934 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:45.233938 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:45.233950 (+ 12us) server_negotiation.cc:407] Connection header received
0926 16:36:45.234027 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:45.234029 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:45.234138 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:45.234240 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:36:45.234494 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:45.235011 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:45.235954 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:45.236178 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:45.236262 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:45.236563 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:36:45.236983 (+ 420us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:36:45.237078 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:45.237165 (+ 87us) server_negotiation.cc:299] Negotiation successful
0926 16:36:45.237215 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":91,"threads_started":1}
I20260926 16:36:45.244609 1473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:45.240533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57480 (local address 127.11.8.253:41685)
0926 16:36:45.240657 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:45.240662 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:45.240860 (+ 198us) server_negotiation.cc:407] Connection header received
0926 16:36:45.241200 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:45.241207 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:45.241318 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:45.241398 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:36:45.241660 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:45.242234 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:45.243384 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:45.243749 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:45.243848 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:45.244098 (+ 250us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:36:45.244186 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:36:45.244248 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:45.244412 (+ 164us) server_negotiation.cc:299] Negotiation successful
0926 16:36:45.244465 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 16:36:45.244948 1479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:45.240926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41284 (local address 127.11.8.252:36607)
0926 16:36:45.241047 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:45.241052 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:45.241185 (+ 133us) server_negotiation.cc:407] Connection header received
0926 16:36:45.241364 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:45.241367 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:45.241423 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:45.241496 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:36:45.241837 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:45.242630 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:45.243643 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:45.243987 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:45.244084 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:45.244490 (+ 406us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:36:45.244560 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:36:45.244604 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:45.244775 (+ 171us) server_negotiation.cc:299] Negotiation successful
0926 16:36:45.244823 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 16:36:45.245540 1486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:45.240467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36540 (local address 127.11.8.254:44201)
0926 16:36:45.241057 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:45.241063 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:45.241079 (+ 16us) server_negotiation.cc:407] Connection header received
0926 16:36:45.241182 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:45.241186 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:45.241358 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:45.241516 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:36:45.241798 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:45.242630 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:45.243765 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:45.244095 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:45.244226 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:45.244542 (+ 316us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:36:45.245106 (+ 564us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:36:45.245203 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:45.245312 (+ 109us) server_negotiation.cc:299] Negotiation successful
0926 16:36:45.245390 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":61,"threads_started":1}
I20260926 16:36:45.252506 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 1119
I20260926 16:36:45.258709 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 1195
I20260926 16:36:45.264365 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 1271
2026-09-26T16:36:45Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3094 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 16:36:45 dist-test-slave-2zvq krb5kdc[1490](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 16:36:45 dist-test-slave-2zvq krb5kdc[1490](info): set up 2 sockets
Sep 26 16:36:45 dist-test-slave-2zvq krb5kdc[1490](info): commencing operation
krb5kdc: starting...
W20260926 16:36:46.316380 11299 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/krb5kdc/test-user.keytab.
Sep 26 16:36:46 dist-test-slave-2zvq krb5kdc[1490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T16:36:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T16:36:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
Entry for principal kudu/127.11.8.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab.
I20260926 16:36:46.438385 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:37221
--webserver_interface=127.11.8.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:36395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:37221,127.11.8.253:39039,127.11.8.252:34881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 16:36:46.544481 1503 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 16:36:46.544795 1503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:46.544866 1503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:46.544935 1503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:46.549091 1503 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:36:46.549191 1503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:46.549232 1503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:46.549269 1503 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:36:46.549299 1503 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:36:46.554527 1503 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:36395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:37221,127.11.8.253:39039,127.11.8.252:34881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.254.keytab
--principal=kudu/127.11.8.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.254:37221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:46.555522 1503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:46.556509 1503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:46.564637 1508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:46.564649 1511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:46.564649 1509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:46.565025 1503 server_base.cc:1034] running on GCE node
I20260926 16:36:46.565560 1503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:46.566640 1503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:46.567835 1503 hybrid_clock.cc:648] HybridClock initialized: now 1790440606567820 us; error 28 us; skew 500 ppm
Sep 26 16:36:46 dist-test-slave-2zvq krb5kdc[1490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440606, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:36:46.570585 1503 init.cc:377] Logged in from keytab as kudu/127.11.8.254@KRBTEST.COM (short username kudu)
I20260926 16:36:46.571403 1503 webserver.cc:460] Webserver started at http://127.11.8.254:39835/ using document root <none> and password file <none>
I20260926 16:36:46.571933 1503 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:46.572021 1503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:46.572242 1503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:46.574229 1503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-0/data/instance:
uuid: "1864eca8a7f248c28b01bc9386739bea"
format_stamp: "Formatted at 2026-09-26 16:36:46 on dist-test-slave-2zvq"
server_key: "d29b225fb76684991ce7f3f251496e14"
server_key_iv: "2141470bd0e4845c0145085c32ab6c54"
server_key_version: "encryptionkey@0"
I20260926 16:36:46.574842 1503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-0/wal/instance:
uuid: "1864eca8a7f248c28b01bc9386739bea"
format_stamp: "Formatted at 2026-09-26 16:36:46 on dist-test-slave-2zvq"
server_key: "d29b225fb76684991ce7f3f251496e14"
server_key_iv: "2141470bd0e4845c0145085c32ab6c54"
server_key_version: "encryptionkey@0"
I20260926 16:36:46.578423 1503 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 16:36:46.580778 1518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:46.582064 1503 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:36:46.582224 1503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-0/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-0/wal
uuid: "1864eca8a7f248c28b01bc9386739bea"
format_stamp: "Formatted at 2026-09-26 16:36:46 on dist-test-slave-2zvq"
server_key: "d29b225fb76684991ce7f3f251496e14"
server_key_iv: "2141470bd0e4845c0145085c32ab6c54"
server_key_version: "encryptionkey@0"
I20260926 16:36:46.582363 1503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:46.593936 1503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:46.597074 1503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:46.597272 1503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:46.603863 1503 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.254:37221
I20260926 16:36:46.604142 1570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.254:37221 every 8 connection(s)
I20260926 16:36:46.604816 1503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-0/data/info.pb
I20260926 16:36:46.607628 1571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:46.611658 1571 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37221 } has no permanent_uuid. Determining permanent_uuid...
I20260926 16:36:46.614066 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 1503
I20260926 16:36:46.614195 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-0/wal/instance
I20260926 16:36:46.614523 11299 external_mini_cluster.cc:1428] Setting key f8b108759d4caeb336cdd9d87b63443e
Sep 26 16:36:46 dist-test-slave-2zvq krb5kdc[1490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440606, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.8.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.253@KRBTEST.COM" created.
I20260926 16:36:46.627166 1574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:46.614917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35236 (local address 127.11.8.254:37221)
0926 16:36:46.615207 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:46.615218 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:46.615264 (+ 46us) server_negotiation.cc:407] Connection header received
0926 16:36:46.615765 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:46.615769 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:46.615962 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:46.616142 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:46.616944 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:46.617813 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:46.618843 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:46.619208 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:46.619394 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:46.621967 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:46.621981 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:46.621991 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:46.622012 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:46.623804 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:46.624293 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:46.624298 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:46.624300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:46.624454 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:46.624764 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:46.624769 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:46.624771 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:46.625254 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:46.625385 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:46.625765 (+ 380us) server_negotiation.cc:299] Negotiation successful
0926 16:36:46.626055 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":12,"server-negotiator.queue_time_us":183,"spinlock_wait_cycles":2304,"thread_start_us":87,"threads_started":1}
I20260926 16:36:46.627166 1573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:46.613918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37221 (local address 127.0.0.1:35236)
0926 16:36:46.614494 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:46.614536 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:46.615472 (+ 936us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:46.616139 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:46.616144 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:46.616576 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:36:46.616803 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:46.616815 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:46.617952 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:46.617955 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:46.618731 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:46.618740 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:46.619366 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:46.619370 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:46.619462 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:46.620047 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:36:46.620062 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:36:46.621800 (+ 1738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:36:46.623943 (+ 2143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:46.623951 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:46.623962 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:46.624184 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:46.624561 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:46.624564 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:46.624566 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:46.624678 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:46.625362 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:36:46.625374 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:36:46.625519 (+ 145us) client_negotiation.cc:769] Sending connection context
0926 16:36:46.625685 (+ 166us) client_negotiation.cc:240] Negotiation successful
0926 16:36:46.626104 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"mutex_wait_us":129,"thread_start_us":149,"threads_started":1}
I20260926 16:36:46.629773 1571 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.253" port: 39039 } has no permanent_uuid. Determining permanent_uuid...
I20260926 16:36:46.630672 1573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:46.630318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.253:39039 (local address 127.0.0.1:43194)
0926 16:36:46.630473 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:46.630574 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.253:39039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260926 16:36:46.630895 1522 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.8.253:39039: connect: Connection refused (error 111)
I20260926 16:36:46.631783 1573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:46.631499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.253:39039 (local address 127.0.0.1:43196)
0926 16:36:46.631633 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:46.631691 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.253:39039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260926 16:36:46.631960 1571 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.253:39039: Network error: Client connection negotiation failed: client connection to 127.11.8.253:39039: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.253.keytab.
Entry for principal kudu/127.11.8.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.253.keytab.
I20260926 16:36:46.641595 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.253:39039
--webserver_interface=127.11.8.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.253.keytab
--principal=kudu/127.11.8.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:36395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:37221,127.11.8.253:39039,127.11.8.252:34881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 16:36:46.681218 1571 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.253" port: 39039 } attempt: 1
I20260926 16:36:46.682188 1573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:46.681845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.253:39039 (local address 127.0.0.1:43206)
0926 16:36:46.682047 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:46.682111 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.253:39039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260926 16:36:46.682956 1573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:46.682723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.253:39039 (local address 127.0.0.1:43212)
0926 16:36:46.682840 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:46.682897 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.253:39039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260926 16:36:46.683072 1571 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.253:39039: Network error: Client connection negotiation failed: client connection to 127.11.8.253:39039: connect: Connection refused (error 111)
W20260926 16:36:46.749836 1578 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 16:36:46.750170 1578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:46.750257 1578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:46.750345 1578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260926 16:36:46.751232 1571 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.253" port: 39039 } attempt: 2
I20260926 16:36:46.752017 1573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:46.751740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.253:39039 (local address 127.0.0.1:43224)
0926 16:36:46.751881 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:46.751946 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.253:39039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260926 16:36:46.752686 1573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:46.752477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.253:39039 (local address 127.0.0.1:43238)
0926 16:36:46.752583 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:46.752627 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.253:39039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260926 16:36:46.752804 1571 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.253:39039: Network error: Client connection negotiation failed: client connection to 127.11.8.253:39039: connect: Connection refused (error 111)
W20260926 16:36:46.754601 1578 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:36:46.754670 1578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:46.754733 1578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:46.754786 1578 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:36:46.754822 1578 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:36:46.759908 1578 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:36395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:37221,127.11.8.253:39039,127.11.8.252:34881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.253.keytab
--principal=kudu/127.11.8.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.253:39039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.8.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1578
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
I20260926 16:36:46.761044 1578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:46.762023 1578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 16:36:46.770123 1583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:46.770115 1584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:46.770318 1578 server_base.cc:1034] running on GCE node
W20260926 16:36:46.770115 1587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:46.770906 1578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:46.771992 1578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:46.773175 1578 hybrid_clock.cc:648] HybridClock initialized: now 1790440606773161 us; error 25 us; skew 500 ppm
Sep 26 16:36:46 dist-test-slave-2zvq krb5kdc[1490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440606, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:36:46.775830 1578 init.cc:377] Logged in from keytab as kudu/127.11.8.253@KRBTEST.COM (short username kudu)
I20260926 16:36:46.776664 1578 webserver.cc:460] Webserver started at http://127.11.8.253:45565/ using document root <none> and password file <none>
I20260926 16:36:46.777154 1578 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:46.777210 1578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:46.777375 1578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:46.779348 1578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-1/data/instance:
uuid: "951d0ac3fd574e489d3482796a673db1"
format_stamp: "Formatted at 2026-09-26 16:36:46 on dist-test-slave-2zvq"
server_key: "c2ab053d9b45b7a7c479f867e2576d77"
server_key_iv: "e90f513581bf9bfd164c2c6c1e383682"
server_key_version: "encryptionkey@0"
I20260926 16:36:46.779855 1578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-1/wal/instance:
uuid: "951d0ac3fd574e489d3482796a673db1"
format_stamp: "Formatted at 2026-09-26 16:36:46 on dist-test-slave-2zvq"
server_key: "c2ab053d9b45b7a7c479f867e2576d77"
server_key_iv: "e90f513581bf9bfd164c2c6c1e383682"
server_key_version: "encryptionkey@0"
I20260926 16:36:46.783289 1578 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 16:36:46.785495 1594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:46.786940 1578 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260926 16:36:46.787099 1578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-1/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-1/wal
uuid: "951d0ac3fd574e489d3482796a673db1"
format_stamp: "Formatted at 2026-09-26 16:36:46 on dist-test-slave-2zvq"
server_key: "c2ab053d9b45b7a7c479f867e2576d77"
server_key_iv: "e90f513581bf9bfd164c2c6c1e383682"
server_key_version: "encryptionkey@0"
I20260926 16:36:46.787199 1578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:46.803217 1578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:46.806047 1578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:46.806239 1578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:46.812979 1578 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.253:39039
I20260926 16:36:46.813259 1646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.253:39039 every 8 connection(s)
I20260926 16:36:46.813941 1578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-1/data/info.pb
I20260926 16:36:46.816670 1647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:46.817557 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 1578
I20260926 16:36:46.817700 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-1/wal/instance
I20260926 16:36:46.817978 11299 external_mini_cluster.cc:1428] Setting key e8812f17b16f9d8dee53d24dc87d475d
I20260926 16:36:46.820962 1647 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37221 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.11.8.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.252@KRBTEST.COM" created.
Sep 26 16:36:46 dist-test-slave-2zvq krb5kdc[1490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440606, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.253@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:36:46.835299 1574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:46.822930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35238 (local address 127.11.8.254:37221)
0926 16:36:46.823066 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:46.823070 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:46.824079 (+ 1009us) server_negotiation.cc:407] Connection header received
0926 16:36:46.824996 (+ 917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:46.825000 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:46.825067 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:46.825154 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:46.826301 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:46.826890 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:46.828082 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:46.828356 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:46.828476 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:46.831203 (+ 2727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:46.831210 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:46.831216 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:46.831252 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:46.833029 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:46.833538 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:46.833545 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:46.833550 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:46.833648 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:46.834037 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:46.834041 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:46.834043 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:46.834335 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:46.834440 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:46.834957 (+ 517us) server_negotiation.cc:299] Negotiation successful
0926 16:36:46.835098 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 16:36:46.836112 1650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:46.823435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37221 (local address 127.0.0.1:35238)
0926 16:36:46.823936 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:46.823976 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:46.824780 (+ 804us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:46.825313 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:46.825321 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:46.825642 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:36:46.826145 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:46.826157 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:46.827012 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:46.827020 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:46.827917 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:46.827926 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:46.828509 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:46.828513 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:46.828673 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:46.829269 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:36:46.829313 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:36:46.831025 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:36:46.833149 (+ 2124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:46.833153 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:46.833164 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:46.833435 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:46.833781 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:46.833783 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:46.833785 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:46.833927 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:46.834453 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:36:46.834458 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:36:46.834688 (+ 230us) client_negotiation.cc:769] Sending connection context
0926 16:36:46.834859 (+ 171us) client_negotiation.cc:240] Negotiation successful
0926 16:36:46.835122 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":137,"threads_started":1}
I20260926 16:36:46.837625 1647 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.253" port: 39039 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 16:36:46 dist-test-slave-2zvq krb5kdc[1490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440606, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.253@KRBTEST.COM for kudu/127.11.8.253@KRBTEST.COM
I20260926 16:36:46.843966 1571 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.253" port: 39039 } attempt: 3
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.252.keytab.
Entry for principal kudu/127.11.8.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.252.keytab.
I20260926 16:36:46.845394 11299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
/tmp/dist-test-taskTzWGcB/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.252:34881
--webserver_interface=127.11.8.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.252.keytab
--principal=kudu/127.11.8.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.212:36395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.254:37221,127.11.8.253:39039,127.11.8.252:34881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 16:36:46.847898 1650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:46.838150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.253:39039 (local address 127.0.0.1:43242)
0926 16:36:46.838241 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:46.838255 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:46.838391 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:46.838953 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:46.838956 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:46.839125 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:36:46.839211 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:46.839245 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:46.840250 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:46.840256 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:46.840878 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:46.840887 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:46.841481 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:46.841485 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:46.841580 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:46.842060 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:36:46.842077 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:36:46.843497 (+ 1420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:36:46.845954 (+ 2457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:46.845957 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:46.845959 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:46.846226 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:46.846587 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:46.846602 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:46.846607 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:46.846719 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:46.847313 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:36:46.847317 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:36:46.847432 (+ 115us) client_negotiation.cc:769] Sending connection context
0926 16:36:46.847540 (+ 108us) client_negotiation.cc:240] Negotiation successful
0926 16:36:46.847688 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260926 16:36:46.848517 1652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:46.838258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43242 (local address 127.11.8.253:39039)
0926 16:36:46.838598 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:46.838606 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:46.838623 (+ 17us) server_negotiation.cc:407] Connection header received
0926 16:36:46.838653 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:46.838657 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:46.838840 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:46.838978 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:46.839371 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:46.840140 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:46.841005 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:46.841366 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:46.841504 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:46.843654 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:46.843666 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:46.843681 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:46.843730 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:46.845844 (+ 2114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:46.846370 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:46.846379 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:46.846384 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:46.846486 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:46.846821 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:46.846823 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:46.846825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:46.847165 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:46.847324 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:46.847679 (+ 355us) server_negotiation.cc:299] Negotiation successful
0926 16:36:46.847842 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":125,"threads_started":1}
Sep 26 16:36:46 dist-test-slave-2zvq krb5kdc[1490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440606, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for kudu/127.11.8.253@KRBTEST.COM
I20260926 16:36:46.850735 1647 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.252" port: 34881 } has no permanent_uuid. Determining permanent_uuid...
I20260926 16:36:46.851526 1650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:46.851204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:34881 (local address 127.0.0.1:53066)
0926 16:36:46.851352 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:46.851442 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.252:34881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260926 16:36:46.851716 1598 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.8.252:34881: connect: Connection refused (error 111)
I20260926 16:36:46.852809 1650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:46.852552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:34881 (local address 127.0.0.1:53068)
0926 16:36:46.852682 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:46.852745 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.252:34881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260926 16:36:46.853441 1647 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.252:34881: Network error: Client connection negotiation failed: client connection to 127.11.8.252:34881: connect: Connection refused (error 111)
I20260926 16:36:46.854650 1653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:46.844693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43246 (local address 127.11.8.253:39039)
0926 16:36:46.845029 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:46.845039 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:46.845055 (+ 16us) server_negotiation.cc:407] Connection header received
0926 16:36:46.845099 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:46.845104 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:46.845149 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:46.845250 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:46.845759 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:46.846563 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:46.848163 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:46.848535 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:46.849027 (+ 492us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:46.850925 (+ 1898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:46.850931 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:46.850936 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:46.850966 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:46.852796 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:46.853342 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:46.853345 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:46.853346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:46.853445 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:46.853755 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:46.853758 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:46.853759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:46.853936 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:46.854012 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:46.854425 (+ 413us) server_negotiation.cc:299] Negotiation successful
0926 16:36:46.854523 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":112,"threads_started":1}
I20260926 16:36:46.854740 1573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:46.844453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.253:39039 (local address 127.0.0.1:43246)
0926 16:36:46.844925 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:46.844942 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:46.845026 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:46.845273 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:46.845276 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:46.845442 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:36:46.845591 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:46.845602 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:46.846694 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:46.846697 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:46.848018 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:46.848032 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:46.848700 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:46.848704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:46.848820 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:46.849327 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:36:46.849341 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:36:46.850746 (+ 1405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:36:46.852915 (+ 2169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:46.852920 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:46.852923 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:46.853212 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:46.853536 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:46.853541 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:46.853544 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:46.853649 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:46.854040 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:36:46.854044 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:36:46.854173 (+ 129us) client_negotiation.cc:769] Sending connection context
0926 16:36:46.854327 (+ 154us) client_negotiation.cc:240] Negotiation successful
0926 16:36:46.854499 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260926 16:36:46.855619 1571 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.252" port: 34881 } has no permanent_uuid. Determining permanent_uuid...
I20260926 16:36:46.856381 1573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:46.856082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:34881 (local address 127.0.0.1:53080)
0926 16:36:46.856226 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:46.856295 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.252:34881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260926 16:36:46.857231 1573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:46.856893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:34881 (local address 127.0.0.1:53090)
0926 16:36:46.857081 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:46.857141 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.252:34881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260926 16:36:46.857375 1571 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.252:34881: Network error: Client connection negotiation failed: client connection to 127.11.8.252:34881: connect: Connection refused (error 111)
I20260926 16:36:46.888538 1571 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.252" port: 34881 } attempt: 1
I20260926 16:36:46.889423 1573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:46.889037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:34881 (local address 127.0.0.1:53094)
0926 16:36:46.889230 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:46.889314 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.252:34881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260926 16:36:46.890257 1573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:46.889978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:34881 (local address 127.0.0.1:53104)
0926 16:36:46.890111 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:46.890177 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.252:34881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260926 16:36:46.890421 1571 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.252:34881: Network error: Client connection negotiation failed: client connection to 127.11.8.252:34881: connect: Connection refused (error 111)
I20260926 16:36:46.902693 1647 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.252" port: 34881 } attempt: 1
I20260926 16:36:46.903445 1650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:46.903187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:34881 (local address 127.0.0.1:53112)
0926 16:36:46.903317 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:46.903377 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.252:34881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260926 16:36:46.904201 1650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:46.903976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:34881 (local address 127.0.0.1:53118)
0926 16:36:46.904099 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:46.904145 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.252:34881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260926 16:36:46.904438 1647 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.252:34881: Network error: Client connection negotiation failed: client connection to 127.11.8.252:34881: connect: Connection refused (error 111)
W20260926 16:36:46.956943 1654 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 16:36:46.957212 1654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 16:36:46.957235 1654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 16:36:46.957329 1654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 16:36:46.961377 1654 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 16:36:46.961426 1654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 16:36:46.961444 1654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 16:36:46.961462 1654 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 16:36:46.961517 1654 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 16:36:46.965590 1571 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.252" port: 34881 } attempt: 2
I20260926 16:36:46.966436 1573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:46.966080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:34881 (local address 127.0.0.1:53134)
0926 16:36:46.966255 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:46.966346 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.252:34881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260926 16:36:46.967151 1573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:46.966950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:34881 (local address 127.0.0.1:53150)
0926 16:36:46.967053 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:46.967097 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.252:34881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260926 16:36:46.967136 1654 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.212:36395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.254:37221,127.11.8.253:39039,127.11.8.252:34881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/krb5kdc/kudu_127.11.8.252.keytab
--principal=kudu/127.11.8.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.252:34881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.8.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 26 Sep 2026 15:58:58 UTC on ccycloud.quasar-ylhwwb.root.comops.site
build id 4674727
W20260926 16:36:46.967434 1571 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.252:34881: Network error: Client connection negotiation failed: client connection to 127.11.8.252:34881: connect: Connection refused (error 111)
I20260926 16:36:46.968185 1654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 16:36:46.969144 1654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 16:36:46.972625 1647 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.252" port: 34881 } attempt: 2
I20260926 16:36:46.973323 1650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:46.973028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:34881 (local address 127.0.0.1:53166)
0926 16:36:46.973185 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:46.973244 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.252:34881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260926 16:36:46.974041 1650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:46.973811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:34881 (local address 127.0.0.1:53178)
0926 16:36:46.973929 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:46.973979 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.252:34881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260926 16:36:46.974181 1647 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.252:34881: Network error: Client connection negotiation failed: client connection to 127.11.8.252:34881: connect: Connection refused (error 111)
W20260926 16:36:46.977368 1660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:46.977417 1661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 16:36:46.977309 1663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 16:36:46.977617 1654 server_base.cc:1034] running on GCE node
I20260926 16:36:46.978169 1654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 16:36:46.979187 1654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 16:36:46.980388 1654 hybrid_clock.cc:648] HybridClock initialized: now 1790440606980377 us; error 49 us; skew 500 ppm
Sep 26 16:36:46 dist-test-slave-2zvq krb5kdc[1490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440606, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 16:36:46.982964 1654 init.cc:377] Logged in from keytab as kudu/127.11.8.252@KRBTEST.COM (short username kudu)
I20260926 16:36:46.983830 1654 webserver.cc:460] Webserver started at http://127.11.8.252:35967/ using document root <none> and password file <none>
I20260926 16:36:46.984360 1654 fs_manager.cc:359] Metadata directory not provided
I20260926 16:36:46.984449 1654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 16:36:46.984665 1654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 16:36:46.986781 1654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-2/data/instance:
uuid: "ccda442fe50246f79120109e76cf2a6a"
format_stamp: "Formatted at 2026-09-26 16:36:46 on dist-test-slave-2zvq"
server_key: "a53b4bb795ab97b137fa4a02038c9c12"
server_key_iv: "55b3a43a9373238f6353f305e76390e6"
server_key_version: "encryptionkey@0"
I20260926 16:36:46.987316 1654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-2/wal/instance:
uuid: "ccda442fe50246f79120109e76cf2a6a"
format_stamp: "Formatted at 2026-09-26 16:36:46 on dist-test-slave-2zvq"
server_key: "a53b4bb795ab97b137fa4a02038c9c12"
server_key_iv: "55b3a43a9373238f6353f305e76390e6"
server_key_version: "encryptionkey@0"
I20260926 16:36:46.990772 1654 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 16:36:46.993031 1670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 16:36:46.994212 1654 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 16:36:46.994392 1654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-2/data,/tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-2/wal
uuid: "ccda442fe50246f79120109e76cf2a6a"
format_stamp: "Formatted at 2026-09-26 16:36:46 on dist-test-slave-2zvq"
server_key: "a53b4bb795ab97b137fa4a02038c9c12"
server_key_iv: "55b3a43a9373238f6353f305e76390e6"
server_key_version: "encryptionkey@0"
I20260926 16:36:46.994503 1654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 16:36:47.016188 1654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 16:36:47.018886 1654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 16:36:47.019076 1654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 16:36:47.025811 1654 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.252:34881
I20260926 16:36:47.026085 1722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.252:34881 every 8 connection(s)
I20260926 16:36:47.026777 1654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-2/data/info.pb
I20260926 16:36:47.029345 1723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 16:36:47.031836 11299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu as pid 1654
I20260926 16:36:47.031988 11299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTzWGcB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790440324823152-11299-0/minicluster-data/master-2/wal/instance
I20260926 16:36:47.032258 11299 external_mini_cluster.cc:1428] Setting key 8f11619dbf81bd9b1dd0602829a6b638
I20260926 16:36:47.033075 1723 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37221 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 16:36:47 dist-test-slave-2zvq krb5kdc[1490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440606, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.252@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
Sep 26 16:36:47 dist-test-slave-2zvq krb5kdc[1490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.252@KRBTEST.COM
I20260926 16:36:47.046393 1574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:47.034902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35244 (local address 127.11.8.254:37221)
0926 16:36:47.035037 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:47.035136 (+ 99us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:47.035234 (+ 98us) server_negotiation.cc:407] Connection header received
0926 16:36:47.035477 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:47.035481 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:47.035531 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:47.035587 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:47.037027 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:47.037783 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:47.039119 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:47.039481 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:47.039586 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:47.042258 (+ 2672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:47.042295 (+ 37us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:47.042301 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:47.042330 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:47.043989 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:47.044657 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:47.044660 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:47.044662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:47.044739 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:47.045125 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:47.045130 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:47.045132 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:47.045343 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:47.045457 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:47.046001 (+ 544us) server_negotiation.cc:299] Negotiation successful
0926 16:36:47.046132 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 16:36:47.047111 1727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:47.034769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.254:37221 (local address 127.0.0.1:35244)
0926 16:36:47.035110 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:47.035156 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:47.035267 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:47.035834 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:47.035844 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:47.036273 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:36:47.036842 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:47.036859 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:47.037920 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:47.037924 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:47.039001 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:47.039010 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:47.039623 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:47.039626 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:47.039736 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:47.040334 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:36:47.040358 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:36:47.042093 (+ 1735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:36:47.044100 (+ 2007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:47.044107 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:47.044117 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:47.044538 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:47.044869 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:47.044873 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:47.044876 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:47.045022 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:47.045454 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:36:47.045460 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:36:47.045732 (+ 272us) client_negotiation.cc:769] Sending connection context
0926 16:36:47.045931 (+ 199us) client_negotiation.cc:240] Negotiation successful
0926 16:36:47.046245 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"mutex_wait_us":27,"spinlock_wait_cycles":768,"thread_start_us":89,"threads_started":1}
I20260926 16:36:47.047109 1726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:47.033577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53188 (local address 127.11.8.252:34881)
0926 16:36:47.034001 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:47.034015 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:47.034047 (+ 32us) server_negotiation.cc:407] Connection header received
0926 16:36:47.034785 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:47.034808 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:47.035245 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:47.036153 (+ 908us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:47.036630 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:47.038037 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:47.039119 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:47.039698 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:47.039837 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:47.042725 (+ 2888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:47.042732 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:47.042746 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:47.042771 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:47.044561 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:47.044996 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:47.045002 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:47.045004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:47.045084 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:47.045377 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:47.045380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:47.045382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:47.045718 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:47.045924 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:47.046108 (+ 184us) server_negotiation.cc:299] Negotiation successful
0926 16:36:47.046339 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":299,"spinlock_wait_cycles":22784,"thread_start_us":127,"threads_started":1}
I20260926 16:36:47.048650 1723 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.253" port: 39039 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 16:36:47 dist-test-slave-2zvq krb5kdc[1490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440606, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.252@KRBTEST.COM for kudu/127.11.8.253@KRBTEST.COM
Sep 26 16:36:47 dist-test-slave-2zvq krb5kdc[1490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.254@KRBTEST.COM
I20260926 16:36:47.057418 1727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:47.049215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.253:39039 (local address 127.0.0.1:43262)
0926 16:36:47.049326 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:47.049383 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:47.049475 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:47.049760 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:47.049763 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:47.049947 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:36:47.050029 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:47.050034 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:47.050851 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:47.050858 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:47.051356 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:47.051363 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:47.051818 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:47.051822 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:47.051915 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:47.052374 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:36:47.052389 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:36:47.053607 (+ 1218us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:36:47.055853 (+ 2246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:47.055855 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:47.055857 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:47.056056 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:47.056363 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:47.056376 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:47.056378 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:47.056466 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:47.056962 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:36:47.056966 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:36:47.057066 (+ 100us) client_negotiation.cc:769] Sending connection context
0926 16:36:47.057145 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 16:36:47.057244 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260926 16:36:47.057781 1653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:47.049398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43262 (local address 127.11.8.253:39039)
0926 16:36:47.049521 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:47.049526 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:47.049540 (+ 14us) server_negotiation.cc:407] Connection header received
0926 16:36:47.049587 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:47.049590 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:47.049639 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:47.049749 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:47.050150 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:47.050745 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:47.051471 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:47.051709 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:47.051818 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:47.053746 (+ 1928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:47.053749 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:47.053751 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:47.053774 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:47.055724 (+ 1950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:47.056154 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:47.056157 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:47.056159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:47.056255 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:47.056622 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:47.056626 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:47.056627 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:47.056858 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:47.056940 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:47.057223 (+ 283us) server_negotiation.cc:299] Negotiation successful
0926 16:36:47.057377 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 16:36:47.058149 1574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:47.049610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35250 (local address 127.11.8.254:37221)
0926 16:36:47.049736 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:47.049740 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:47.049751 (+ 11us) server_negotiation.cc:407] Connection header received
0926 16:36:47.049787 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:47.049789 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:47.049818 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:47.049905 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:47.050528 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:47.050996 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:47.051745 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:47.051974 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:47.052082 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:47.054419 (+ 2337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:47.054424 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:47.054426 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:47.054456 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:47.056153 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:47.056629 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:47.056633 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:47.056635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:47.056713 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:47.057009 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:47.057012 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:47.057013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:47.057223 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:47.057441 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:47.057858 (+ 417us) server_negotiation.cc:299] Negotiation successful
0926 16:36:47.057985 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 16:36:47.058717 1723 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.252" port: 34881 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 16:36:47 dist-test-slave-2zvq krb5kdc[1490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440606, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.252@KRBTEST.COM for kudu/127.11.8.252@KRBTEST.COM
Sep 26 16:36:47 dist-test-slave-2zvq krb5kdc[1490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.253@KRBTEST.COM
I20260926 16:36:47.065374 1647 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.252" port: 34881 } attempt: 3
I20260926 16:36:47.066606 1571 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.252" port: 34881 } attempt: 3
I20260926 16:36:47.067654 1727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:47.059204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:34881 (local address 127.0.0.1:53200)
0926 16:36:47.059303 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:47.059318 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:47.059389 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:47.059756 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:47.059759 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:47.059899 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:36:47.059962 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:47.059967 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:47.060752 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:47.060755 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:47.061328 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:47.061337 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:47.061878 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:47.061882 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:47.061964 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:47.062475 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:36:47.062485 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:36:47.063784 (+ 1299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:36:47.065575 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:47.065578 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:47.065580 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:47.065821 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:47.066109 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:47.066112 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:47.066114 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:47.066189 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:47.066836 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:36:47.066842 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:36:47.066943 (+ 101us) client_negotiation.cc:769] Sending connection context
0926 16:36:47.067302 (+ 359us) client_negotiation.cc:240] Negotiation successful
0926 16:36:47.067459 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260926 16:36:47.068112 1726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:47.059407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53200 (local address 127.11.8.252:34881)
0926 16:36:47.059537 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:47.059543 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:47.059557 (+ 14us) server_negotiation.cc:407] Connection header received
0926 16:36:47.059588 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:47.059591 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:47.059626 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:47.059720 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:47.060072 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:47.060654 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:47.061456 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:47.061745 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:47.061858 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:47.063923 (+ 2065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:47.063927 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:47.063930 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:47.063953 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:47.065464 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:47.065934 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:47.065937 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:47.065938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:47.066014 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:47.066300 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:47.066304 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:47.066305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:47.066507 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:47.066644 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:47.067884 (+ 1240us) server_negotiation.cc:299] Negotiation successful
0926 16:36:47.067983 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 16:36:47.068575 1653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:47.059269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43276 (local address 127.11.8.253:39039)
0926 16:36:47.059361 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:47.059388 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:47.059441 (+ 53us) server_negotiation.cc:407] Connection header received
0926 16:36:47.059565 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:47.059568 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:47.059602 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:47.059660 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:47.060320 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:47.060824 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:47.061549 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:47.061745 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:47.061851 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:47.064538 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:47.064542 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:47.064544 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:47.064567 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:47.066346 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:47.066822 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:47.066827 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:47.066831 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:47.066910 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:47.067238 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:47.067242 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:47.067243 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:47.067457 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:47.067670 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:47.068309 (+ 639us) server_negotiation.cc:299] Negotiation successful
0926 16:36:47.068426 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260926 16:36:47.071151 1723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccda442fe50246f79120109e76cf2a6a: Bootstrap starting.
Sep 26 16:36:47 dist-test-slave-2zvq krb5kdc[1490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440606, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.253@KRBTEST.COM for kudu/127.11.8.252@KRBTEST.COM
I20260926 16:36:47.072463 11299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 16:36:47.074014 1723 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ccda442fe50246f79120109e76cf2a6a: Neither blocks nor log segments found. Creating new log.
Sep 26 16:36:47 dist-test-slave-2zvq krb5kdc[1490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790440606, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.254@KRBTEST.COM for kudu/127.11.8.252@KRBTEST.COM
I20260926 16:36:47.074914 1723 log.cc:824] T 00000000000000000000000000000000 P ccda442fe50246f79120109e76cf2a6a: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:47.073611 1545 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "1864eca8a7f248c28b01bc9386739bea"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35250
I20260926 16:36:47.076640 1728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:47.065859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53216 (local address 127.11.8.252:34881)
0926 16:36:47.066186 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:47.066195 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:47.066206 (+ 11us) server_negotiation.cc:407] Connection header received
0926 16:36:47.066239 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:47.066245 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:47.066316 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:47.066403 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:47.067113 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:47.068961 (+ 1848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:47.069716 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:47.070051 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:47.070150 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:47.072188 (+ 2038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:47.072192 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:47.072194 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:47.072215 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:47.074732 (+ 2517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:47.075498 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:47.075501 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:47.075503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:47.075580 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:47.075934 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:47.075940 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:47.075944 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:47.076127 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:47.076201 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:47.076450 (+ 249us) server_negotiation.cc:299] Negotiation successful
0926 16:36:47.076528 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"spinlock_wait_cycles":1178496,"thread_start_us":143,"threads_started":1}
I20260926 16:36:47.076674 1650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:47.065765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:34881 (local address 127.0.0.1:53216)
0926 16:36:47.065938 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:47.065952 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:47.066024 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:47.066430 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:47.066434 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:47.066600 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:36:47.066729 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:47.066739 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:47.069071 (+ 2332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:47.069076 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:47.069602 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:47.069609 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:47.070150 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:47.070154 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:47.070233 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:47.070721 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:36:47.070735 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:36:47.072068 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:36:47.074840 (+ 2772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:47.074846 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:47.074850 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:47.075093 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:47.075713 (+ 620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:47.075719 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:47.075723 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:47.075816 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:47.076211 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:36:47.076213 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:36:47.076302 (+ 89us) client_negotiation.cc:769] Sending connection context
0926 16:36:47.076409 (+ 107us) client_negotiation.cc:240] Negotiation successful
0926 16:36:47.076538 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107}
I20260926 16:36:47.077183 1723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccda442fe50246f79120109e76cf2a6a: No bootstrap required, opened a new log
I20260926 16:36:47.079123 1647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 951d0ac3fd574e489d3482796a673db1: Bootstrap starting.
I20260926 16:36:47.081144 1647 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 951d0ac3fd574e489d3482796a673db1: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:47.081189 1723 raft_consensus.cc:348] T 00000000000000000000000000000000 P ccda442fe50246f79120109e76cf2a6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1864eca8a7f248c28b01bc9386739bea" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37221 } } peers { permanent_uuid: "951d0ac3fd574e489d3482796a673db1" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 39039 } } peers { permanent_uuid: "ccda442fe50246f79120109e76cf2a6a" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 34881 } }
I20260926 16:36:47.081441 1723 raft_consensus.cc:729] T 00000000000000000000000000000000 P ccda442fe50246f79120109e76cf2a6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccda442fe50246f79120109e76cf2a6a, State: Initialized, Role: FOLLOWER
I20260926 16:36:47.081769 1723 consensus_queue.cc:260] T 00000000000000000000000000000000 P ccda442fe50246f79120109e76cf2a6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1864eca8a7f248c28b01bc9386739bea" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37221 } } peers { permanent_uuid: "951d0ac3fd574e489d3482796a673db1" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 39039 } } peers { permanent_uuid: "ccda442fe50246f79120109e76cf2a6a" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 34881 } }
I20260926 16:36:47.082015 1647 log.cc:824] T 00000000000000000000000000000000 P 951d0ac3fd574e489d3482796a673db1: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:47.082690 1731 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccda442fe50246f79120109e76cf2a6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1864eca8a7f248c28b01bc9386739bea" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37221 } } peers { permanent_uuid: "951d0ac3fd574e489d3482796a673db1" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 39039 } } peers { permanent_uuid: "ccda442fe50246f79120109e76cf2a6a" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 34881 } } }
I20260926 16:36:47.082998 1731 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccda442fe50246f79120109e76cf2a6a [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:36:47.083135 1723 sys_catalog.cc:564] T 00000000000000000000000000000000 P ccda442fe50246f79120109e76cf2a6a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:36:47.083809 1573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:47.066976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.252:34881 (local address 127.0.0.1:53230)
0926 16:36:47.067101 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0926 16:36:47.067115 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 16:36:47.067183 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 16:36:47.067577 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 16:36:47.067581 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 16:36:47.067759 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0926 16:36:47.067874 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:47.067885 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:47.070882 (+ 2997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:47.070888 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:47.071721 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 16:36:47.071733 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 16:36:47.072522 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 16:36:47.072526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 16:36:47.072645 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:47.073048 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 16:36:47.073067 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 16:36:47.074583 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 16:36:47.082117 (+ 7534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:47.082121 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:47.082126 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:47.082422 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:47.082755 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 16:36:47.082758 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 16:36:47.082760 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 16:36:47.082850 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 16:36:47.083294 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 16:36:47.083297 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 16:36:47.083402 (+ 105us) client_negotiation.cc:769] Sending connection context
0926 16:36:47.083489 (+ 87us) client_negotiation.cc:240] Negotiation successful
0926 16:36:47.083667 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260926 16:36:47.084128 1647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 951d0ac3fd574e489d3482796a673db1: No bootstrap required, opened a new log
I20260926 16:36:47.084481 1729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:47.067027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53230 (local address 127.11.8.252:34881)
0926 16:36:47.067343 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:47.067352 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:47.067365 (+ 13us) server_negotiation.cc:407] Connection header received
0926 16:36:47.067395 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:47.067398 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:47.067436 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:47.067984 (+ 548us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:47.068121 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:47.070742 (+ 2621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:47.072039 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:47.072383 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:47.072947 (+ 564us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:47.074754 (+ 1807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:47.074760 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:47.074766 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:47.074792 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:47.081981 (+ 7189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:47.082551 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:47.082554 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:47.082556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:47.082640 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:47.082978 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:47.082981 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:47.082982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:47.083185 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:47.084080 (+ 895us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:47.084155 (+ 75us) server_negotiation.cc:299] Negotiation successful
0926 16:36:47.084263 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":103,"threads_started":1}
I20260926 16:36:47.087483 1571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1864eca8a7f248c28b01bc9386739bea: Bootstrap starting.
I20260926 16:36:47.088073 1647 raft_consensus.cc:348] T 00000000000000000000000000000000 P 951d0ac3fd574e489d3482796a673db1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1864eca8a7f248c28b01bc9386739bea" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37221 } } peers { permanent_uuid: "951d0ac3fd574e489d3482796a673db1" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 39039 } } peers { permanent_uuid: "ccda442fe50246f79120109e76cf2a6a" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 34881 } }
W20260926 16:36:47.088213 1743 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ccda442fe50246f79120109e76cf2a6a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 16:36:47.088270 1743 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 16:36:47.088284 1647 raft_consensus.cc:729] T 00000000000000000000000000000000 P 951d0ac3fd574e489d3482796a673db1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 951d0ac3fd574e489d3482796a673db1, State: Initialized, Role: FOLLOWER
I20260926 16:36:47.088593 1647 consensus_queue.cc:260] T 00000000000000000000000000000000 P 951d0ac3fd574e489d3482796a673db1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1864eca8a7f248c28b01bc9386739bea" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37221 } } peers { permanent_uuid: "951d0ac3fd574e489d3482796a673db1" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 39039 } } peers { permanent_uuid: "ccda442fe50246f79120109e76cf2a6a" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 34881 } }
I20260926 16:36:47.089778 1647 sys_catalog.cc:564] T 00000000000000000000000000000000 P 951d0ac3fd574e489d3482796a673db1 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:36:47.090046 1571 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1864eca8a7f248c28b01bc9386739bea: Neither blocks nor log segments found. Creating new log.
I20260926 16:36:47.089135 1745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 951d0ac3fd574e489d3482796a673db1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1864eca8a7f248c28b01bc9386739bea" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37221 } } peers { permanent_uuid: "951d0ac3fd574e489d3482796a673db1" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 39039 } } peers { permanent_uuid: "ccda442fe50246f79120109e76cf2a6a" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 34881 } } }
I20260926 16:36:47.090345 1745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 951d0ac3fd574e489d3482796a673db1 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:36:47.091576 1571 log.cc:824] T 00000000000000000000000000000000 P 1864eca8a7f248c28b01bc9386739bea: Log is configured to *not* fsync() on all Append() calls
I20260926 16:36:47.093497 1571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1864eca8a7f248c28b01bc9386739bea: No bootstrap required, opened a new log
I20260926 16:36:47.091230 1654 master_runner.cc:186] Error getting master registration for 127.11.8.254:37221: OK, instance_id { permanent_uuid: "1864eca8a7f248c28b01bc9386739bea" instance_seqno: 1790440606601931 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 16:36:47.094588 1654 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 16:36:47.093094 1578 master_runner.cc:186] Error getting master registration for 127.11.8.254:37221: OK, instance_id { permanent_uuid: "1864eca8a7f248c28b01bc9386739bea" instance_seqno: 1790440606601931 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 16:36:47.096443 1578 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 16:36:47.096448 1571 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1864eca8a7f248c28b01bc9386739bea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1864eca8a7f248c28b01bc9386739bea" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37221 } } peers { permanent_uuid: "951d0ac3fd574e489d3482796a673db1" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 39039 } } peers { permanent_uuid: "ccda442fe50246f79120109e76cf2a6a" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 34881 } }
I20260926 16:36:47.096674 1571 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1864eca8a7f248c28b01bc9386739bea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1864eca8a7f248c28b01bc9386739bea, State: Initialized, Role: FOLLOWER
I20260926 16:36:47.097008 1571 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1864eca8a7f248c28b01bc9386739bea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1864eca8a7f248c28b01bc9386739bea" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37221 } } peers { permanent_uuid: "951d0ac3fd574e489d3482796a673db1" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 39039 } } peers { permanent_uuid: "ccda442fe50246f79120109e76cf2a6a" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 34881 } }
I20260926 16:36:47.097530 1758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1864eca8a7f248c28b01bc9386739bea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1864eca8a7f248c28b01bc9386739bea" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37221 } } peers { permanent_uuid: "951d0ac3fd574e489d3482796a673db1" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 39039 } } peers { permanent_uuid: "ccda442fe50246f79120109e76cf2a6a" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 34881 } } }
I20260926 16:36:47.097754 1758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1864eca8a7f248c28b01bc9386739bea [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:36:47.098354 1571 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1864eca8a7f248c28b01bc9386739bea [sys.catalog]: configured and running, proceeding with master startup.
I20260926 16:36:47.098400 1545 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1864eca8a7f248c28b01bc9386739bea [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260926 16:36:47.098488 1545 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1864eca8a7f248c28b01bc9386739bea [term 0 FOLLOWER]: Advancing to term 1
W20260926 16:36:47.099680 1756 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 951d0ac3fd574e489d3482796a673db1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 16:36:47.099740 1756 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 16:36:47.099879 1545 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1864eca8a7f248c28b01bc9386739bea [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1864eca8a7f248c28b01bc9386739bea" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37221 } } peers { permanent_uuid: "951d0ac3fd574e489d3482796a673db1" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 39039 } } peers { permanent_uuid: "ccda442fe50246f79120109e76cf2a6a" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 34881 } }
I20260926 16:36:47.100844 1545 leader_election.cc:288] T 00000000000000000000000000000000 P 1864eca8a7f248c28b01bc9386739bea [CANDIDATE]: Term 1 election: Requested vote from peers 951d0ac3fd574e489d3482796a673db1 (127.11.8.253:39039), ccda442fe50246f79120109e76cf2a6a (127.11.8.252:34881)
W20260926 16:36:47.101135 1524 tablet.cc:2369] T 00000000000000000000000000000000 P 1864eca8a7f248c28b01bc9386739bea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 16:36:47.102088 1621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1864eca8a7f248c28b01bc9386739bea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "951d0ac3fd574e489d3482796a673db1"
I20260926 16:36:47.102342 1621 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 951d0ac3fd574e489d3482796a673db1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:47.102442 1697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1864eca8a7f248c28b01bc9386739bea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ccda442fe50246f79120109e76cf2a6a"
I20260926 16:36:47.102674 1697 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ccda442fe50246f79120109e76cf2a6a [term 0 FOLLOWER]: Advancing to term 1
I20260926 16:36:47.103681 1621 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 951d0ac3fd574e489d3482796a673db1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1864eca8a7f248c28b01bc9386739bea in term 1.
I20260926 16:36:47.103991 1697 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ccda442fe50246f79120109e76cf2a6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1864eca8a7f248c28b01bc9386739bea in term 1.
I20260926 16:36:47.104401 1522 leader_election.cc:302] T 00000000000000000000000000000000 P 1864eca8a7f248c28b01bc9386739bea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1864eca8a7f248c28b01bc9386739bea, ccda442fe50246f79120109e76cf2a6a; no voters:
W20260926 16:36:47.105784 1770 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1864eca8a7f248c28b01bc9386739bea: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 16:36:47.105872 1770 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 16:36:47.108817 1758 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1864eca8a7f248c28b01bc9386739bea [term 1 FOLLOWER]: Leader election won for term 1
I20260926 16:36:47.109124 1758 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1864eca8a7f248c28b01bc9386739bea [term 1 LEADER]: Becoming Leader. State: Replica: 1864eca8a7f248c28b01bc9386739bea, State: Running, Role: LEADER
I20260926 16:36:47.109459 1503 master_runner.cc:263] No leader master found from master 1864eca8a7f248c28b01bc9386739bea
I20260926 16:36:47.109545 1503 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 16:36:47.109455 1758 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1864eca8a7f248c28b01bc9386739bea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1864eca8a7f248c28b01bc9386739bea" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37221 } } peers { permanent_uuid: "951d0ac3fd574e489d3482796a673db1" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 39039 } } peers { permanent_uuid: "ccda442fe50246f79120109e76cf2a6a" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 34881 } }
I20260926 16:36:47.111193 1776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1864eca8a7f248c28b01bc9386739bea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1864eca8a7f248c28b01bc9386739bea. Latest consensus state: current_term: 1 leader_uuid: "1864eca8a7f248c28b01bc9386739bea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1864eca8a7f248c28b01bc9386739bea" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37221 } } peers { permanent_uuid: "951d0ac3fd574e489d3482796a673db1" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 39039 } } peers { permanent_uuid: "ccda442fe50246f79120109e76cf2a6a" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 34881 } } }
I20260926 16:36:47.111310 1776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1864eca8a7f248c28b01bc9386739bea [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:47.127339 1574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:47.111933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35252 (local address 127.11.8.254:37221)
0926 16:36:47.112041 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:47.112046 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:47.113674 (+ 1628us) server_negotiation.cc:407] Connection header received
0926 16:36:47.113849 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:47.113852 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:47.113898 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:47.113946 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:47.114951 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:47.115737 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:47.116924 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:47.117257 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:47.117325 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:47.122610 (+ 5285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:47.122614 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:47.122617 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:47.122643 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:47.124399 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:47.124876 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:47.124880 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:47.124882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:47.124962 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:47.125241 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:47.125244 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:47.125246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:47.125457 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:47.125572 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:47.127048 (+ 1476us) server_negotiation.cc:299] Negotiation successful
0926 16:36:47.127167 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 16:36:47.132174 1778 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 16:36:47.133208 1778 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 16:36:47.134073 1729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:47.113140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53242 (local address 127.11.8.252:34881)
0926 16:36:47.113239 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:47.113243 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:47.119566 (+ 6323us) server_negotiation.cc:407] Connection header received
0926 16:36:47.119726 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:47.119730 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:47.119773 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:47.119823 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:47.120610 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:47.121422 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:47.125873 (+ 4451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:47.126348 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:47.126425 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:47.130238 (+ 3813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:47.130242 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:47.130245 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:47.130305 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:47.131989 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:47.132484 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:47.132487 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:47.132489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:47.132574 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:47.132856 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:47.132859 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:47.132861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:47.133062 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:47.133160 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:47.133521 (+ 361us) server_negotiation.cc:299] Negotiation successful
0926 16:36:47.133645 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 16:36:47.136898 1653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:47.112587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43290 (local address 127.11.8.253:39039)
0926 16:36:47.112689 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:47.112693 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:47.118522 (+ 5829us) server_negotiation.cc:407] Connection header received
0926 16:36:47.118689 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:47.118693 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:47.118736 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:47.118784 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:47.126362 (+ 7578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:47.127160 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:47.128629 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:47.128985 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:47.129055 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:47.133408 (+ 4353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 16:36:47.133412 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 16:36:47.133414 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 16:36:47.133443 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 16:36:47.135123 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:47.135588 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:47.135592 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:47.135594 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:47.135674 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 16:36:47.135972 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 16:36:47.135976 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 16:36:47.135978 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 16:36:47.136180 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 16:36:47.136288 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:47.136618 (+ 330us) server_negotiation.cc:299] Negotiation successful
0926 16:36:47.136734 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 16:36:47.145040 1697 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ccda442fe50246f79120109e76cf2a6a [term 1 FOLLOWER]: Refusing update from remote peer 1864eca8a7f248c28b01bc9386739bea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:36:47.145746 1776 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1864eca8a7f248c28b01bc9386739bea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccda442fe50246f79120109e76cf2a6a" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 34881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:36:47.151440 1731 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccda442fe50246f79120109e76cf2a6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1864eca8a7f248c28b01bc9386739bea. Latest consensus state: current_term: 1 leader_uuid: "1864eca8a7f248c28b01bc9386739bea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1864eca8a7f248c28b01bc9386739bea" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37221 } } peers { permanent_uuid: "951d0ac3fd574e489d3482796a673db1" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 39039 } } peers { permanent_uuid: "ccda442fe50246f79120109e76cf2a6a" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 34881 } } }
I20260926 16:36:47.151553 1731 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccda442fe50246f79120109e76cf2a6a [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:36:47.152560 1621 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 951d0ac3fd574e489d3482796a673db1 [term 1 FOLLOWER]: Refusing update from remote peer 1864eca8a7f248c28b01bc9386739bea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 16:36:47.153170 1776 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1864eca8a7f248c28b01bc9386739bea [LEADER]: Connected to new peer: Peer: permanent_uuid: "951d0ac3fd574e489d3482796a673db1" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 39039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 16:36:47.157002 1745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 951d0ac3fd574e489d3482796a673db1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1864eca8a7f248c28b01bc9386739bea. Latest consensus state: current_term: 1 leader_uuid: "1864eca8a7f248c28b01bc9386739bea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1864eca8a7f248c28b01bc9386739bea" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37221 } } peers { permanent_uuid: "951d0ac3fd574e489d3482796a673db1" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 39039 } } peers { permanent_uuid: "ccda442fe50246f79120109e76cf2a6a" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 34881 } } }
I20260926 16:36:47.157114 1745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 951d0ac3fd574e489d3482796a673db1 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:36:47.164597 1776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1864eca8a7f248c28b01bc9386739bea [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1864eca8a7f248c28b01bc9386739bea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1864eca8a7f248c28b01bc9386739bea" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37221 } } peers { permanent_uuid: "951d0ac3fd574e489d3482796a673db1" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 39039 } } peers { permanent_uuid: "ccda442fe50246f79120109e76cf2a6a" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 34881 } } }
I20260926 16:36:47.164716 1776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1864eca8a7f248c28b01bc9386739bea [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:47.165926 1745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 951d0ac3fd574e489d3482796a673db1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1864eca8a7f248c28b01bc9386739bea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1864eca8a7f248c28b01bc9386739bea" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37221 } } peers { permanent_uuid: "951d0ac3fd574e489d3482796a673db1" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 39039 } } peers { permanent_uuid: "ccda442fe50246f79120109e76cf2a6a" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 34881 } } }
I20260926 16:36:47.166028 1745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 951d0ac3fd574e489d3482796a673db1 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:36:47.166488 1782 mvcc.cc:205] Tried to move back new op lower bound from 7333644726846889984 to 7333644726723596288. Current Snapshot: MvccSnapshot[applied={T|T < 7333644726846889984}]
I20260926 16:36:47.169369 1785 mvcc.cc:205] Tried to move back new op lower bound from 7333644726846889984 to 7333644726723596288. Current Snapshot: MvccSnapshot[applied={T|T < 7333644726846889984}]
I20260926 16:36:47.170029 1758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1864eca8a7f248c28b01bc9386739bea [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1864eca8a7f248c28b01bc9386739bea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1864eca8a7f248c28b01bc9386739bea" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37221 } } peers { permanent_uuid: "951d0ac3fd574e489d3482796a673db1" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 39039 } } peers { permanent_uuid: "ccda442fe50246f79120109e76cf2a6a" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 34881 } } }
I20260926 16:36:47.170153 1758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1864eca8a7f248c28b01bc9386739bea [sys.catalog]: This master's current role is: LEADER
I20260926 16:36:47.171067 1731 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccda442fe50246f79120109e76cf2a6a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1864eca8a7f248c28b01bc9386739bea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1864eca8a7f248c28b01bc9386739bea" member_type: VOTER last_known_addr { host: "127.11.8.254" port: 37221 } } peers { permanent_uuid: "951d0ac3fd574e489d3482796a673db1" member_type: VOTER last_known_addr { host: "127.11.8.253" port: 39039 } } peers { permanent_uuid: "ccda442fe50246f79120109e76cf2a6a" member_type: VOTER last_known_addr { host: "127.11.8.252" port: 34881 } } }
I20260926 16:36:47.171183 1731 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccda442fe50246f79120109e76cf2a6a [sys.catalog]: This master's current role is: FOLLOWER
I20260926 16:36:47.173278 1783 mvcc.cc:205] Tried to move back new op lower bound from 7333644726846889984 to 7333644726723596288. Current Snapshot: MvccSnapshot[applied={T|T < 7333644726846889984}]
I20260926 16:36:47.178551 1778 catalog_manager.cc:1347] Generated new cluster ID: bdca233ce62f456082d8ff2003664c90
I20260926 16:36:47.178610 1778 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 16:36:47.202164 1778 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 16:36:47.204877 1778 catalog_manager.cc:1504] Loading token signing keys...
I20260926 16:36:47.220921 1778 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1864eca8a7f248c28b01bc9386739bea: Generated new TSK 0
I20260926 16:36:47.221755 1778 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 16:36:47.248057 1574 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:36:47.244699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35254 (local address 127.11.8.254:37221)
0926 16:36:47.244849 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:47.244855 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:47.246719 (+ 1864us) server_negotiation.cc:407] Connection header received
0926 16:36:47.247161 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:47.247165 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:47.247219 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:47.247275 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:47.247919 (+ 644us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35254: BlockingRecv error: recv got EOF from 127.0.0.1:35254 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260926 16:36:47.249353 1653 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:36:47.245438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43300 (local address 127.11.8.253:39039)
0926 16:36:47.245556 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:47.245563 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:47.247077 (+ 1514us) server_negotiation.cc:407] Connection header received
0926 16:36:47.248585 (+ 1508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:47.248588 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:47.248638 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:47.248690 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:47.249231 (+ 541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43300: BlockingRecv error: recv got EOF from 127.0.0.1:43300 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260926 16:36:47.251140 1729 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:36:47.246054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53246 (local address 127.11.8.252:34881)
0926 16:36:47.246204 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:47.246208 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:47.250005 (+ 3797us) server_negotiation.cc:407] Connection header received
0926 16:36:47.250400 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:47.250403 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:47.250456 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:47.250506 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:47.251030 (+ 524us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53246: BlockingRecv error: recv got EOF from 127.0.0.1:53246 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260926 16:36:47.255722 1653 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:36:47.253702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43310 (local address 127.11.8.253:39039)
0926 16:36:47.253833 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:47.253837 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:47.254795 (+ 958us) server_negotiation.cc:407] Connection header received
0926 16:36:47.255019 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:47.255022 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:47.255094 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:47.255144 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:47.255626 (+ 482us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43310: BlockingRecv error: recv got EOF from 127.0.0.1:43310 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260926 16:36:47.260807 1653 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:36:47.259343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43324 (local address 127.11.8.253:39039)
0926 16:36:47.259439 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:47.259443 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:47.259961 (+ 518us) server_negotiation.cc:407] Connection header received
0926 16:36:47.260139 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:47.260142 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:47.260191 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:47.260241 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:47.260722 (+ 481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43324: BlockingRecv error: recv got EOF from 127.0.0.1:43324 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260926 16:36:47.266189 1653 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:36:47.264737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43328 (local address 127.11.8.253:39039)
0926 16:36:47.264826 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:47.264843 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:47.265335 (+ 492us) server_negotiation.cc:407] Connection header received
0926 16:36:47.265516 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:47.265519 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:47.265565 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:47.265616 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:47.266107 (+ 491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43328: BlockingRecv error: recv got EOF from 127.0.0.1:43328 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260926 16:36:47.274331 1653 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:36:47.272647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43340 (local address 127.11.8.253:39039)
0926 16:36:47.272779 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:47.272784 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:47.273380 (+ 596us) server_negotiation.cc:407] Connection header received
0926 16:36:47.273576 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:47.273580 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:47.273632 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:47.273685 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:47.274212 (+ 527us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43340: BlockingRecv error: recv got EOF from 127.0.0.1:43340 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260926 16:36:47.286109 1653 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:36:47.284453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43348 (local address 127.11.8.253:39039)
0926 16:36:47.284557 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:47.284561 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:47.285128 (+ 567us) server_negotiation.cc:407] Connection header received
0926 16:36:47.285311 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:47.285314 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:47.285383 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:47.285432 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:47.286008 (+ 576us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43348: BlockingRecv error: recv got EOF from 127.0.0.1:43348 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260926 16:36:47.305480 1653 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:36:47.304236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43352 (local address 127.11.8.253:39039)
0926 16:36:47.304353 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:47.304358 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:47.304618 (+ 260us) server_negotiation.cc:407] Connection header received
0926 16:36:47.304795 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:47.304799 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:47.304857 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:47.304944 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:47.305370 (+ 426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43352: BlockingRecv error: recv got EOF from 127.0.0.1:43352 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260926 16:36:47.340360 1653 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:36:47.339350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43358 (local address 127.11.8.253:39039)
0926 16:36:47.339492 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:47.339496 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:47.339625 (+ 129us) server_negotiation.cc:407] Connection header received
0926 16:36:47.339744 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:47.339746 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:47.339805 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:47.339887 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:47.340269 (+ 382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43358: BlockingRecv error: recv got EOF from 127.0.0.1:43358 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260926 16:36:47.407779 1653 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:36:47.406676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44196 (local address 127.11.8.253:39039)
0926 16:36:47.406815 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:47.406819 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:47.406982 (+ 163us) server_negotiation.cc:407] Connection header received
0926 16:36:47.407103 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:47.407106 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:47.407152 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:47.407235 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:47.407682 (+ 447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44196: BlockingRecv error: recv got EOF from 127.0.0.1:44196 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260926 16:36:47.539137 1653 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:36:47.537874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44206 (local address 127.11.8.253:39039)
0926 16:36:47.538075 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:47.538079 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:47.538151 (+ 72us) server_negotiation.cc:407] Connection header received
0926 16:36:47.538302 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:47.538305 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:47.538409 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:47.538490 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:47.538993 (+ 503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44206: BlockingRecv error: recv got EOF from 127.0.0.1:44206 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260926 16:36:47.798247 1653 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 16:36:47.796994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44212 (local address 127.11.8.253:39039)
0926 16:36:47.797151 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:47.797156 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:47.797278 (+ 122us) server_negotiation.cc:407] Connection header received
0926 16:36:47.797448 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:47.797451 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:47.797508 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:47.797591 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 16:36:47.798139 (+ 548us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44212: BlockingRecv error: recv got EOF from 127.0.0.1:44212 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 16:36:48.089643 1743 catalog_manager.cc:1259] Loaded cluster ID: bdca233ce62f456082d8ff2003664c90
I20260926 16:36:48.089723 1743 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ccda442fe50246f79120109e76cf2a6a: loading cluster ID for follower catalog manager: success
I20260926 16:36:48.091609 1743 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ccda442fe50246f79120109e76cf2a6a: acquiring CA information for follower catalog manager: success
I20260926 16:36:48.093392 1743 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ccda442fe50246f79120109e76cf2a6a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 16:36:48.101094 1756 catalog_manager.cc:1259] Loaded cluster ID: bdca233ce62f456082d8ff2003664c90
I20260926 16:36:48.101157 1756 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 951d0ac3fd574e489d3482796a673db1: loading cluster ID for follower catalog manager: success
I20260926 16:36:48.102856 1756 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 951d0ac3fd574e489d3482796a673db1: acquiring CA information for follower catalog manager: success
I20260926 16:36:48.103966 1756 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 951d0ac3fd574e489d3482796a673db1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 16:36:48.316519 1857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:48.312319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44214 (local address 127.11.8.253:39039)
0926 16:36:48.312672 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:48.312677 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:48.312787 (+ 110us) server_negotiation.cc:407] Connection header received
0926 16:36:48.312984 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:48.312992 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:48.313115 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:48.313236 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:36:48.313501 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:48.314031 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:48.314995 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:48.315234 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:48.315390 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:48.315698 (+ 308us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:36:48.316122 (+ 424us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:36:48.316257 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:48.316297 (+ 40us) server_negotiation.cc:299] Negotiation successful
0926 16:36:48.316360 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":84,"threads_started":1}
I20260926 16:36:48.323158 1863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:48.319620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35410 (local address 127.11.8.252:34881)
0926 16:36:48.319848 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:48.319854 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:48.319881 (+ 27us) server_negotiation.cc:407] Connection header received
0926 16:36:48.319971 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:48.319973 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:48.320079 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:48.320201 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:36:48.320416 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:48.320926 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:48.321739 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:48.321974 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:48.322080 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:48.322382 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:36:48.322771 (+ 389us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:36:48.322914 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:48.322948 (+ 34us) server_negotiation.cc:299] Negotiation successful
0926 16:36:48.323001 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":58,"threads_started":1}
I20260926 16:36:48.330363 1857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:48.326331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44226 (local address 127.11.8.253:39039)
0926 16:36:48.326459 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:48.326464 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:48.326642 (+ 178us) server_negotiation.cc:407] Connection header received
0926 16:36:48.326849 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:48.326852 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:48.326938 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:48.326992 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:36:48.327343 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:48.328088 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:48.329080 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:48.329416 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:48.329517 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:48.329836 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:36:48.329935 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:36:48.329979 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:48.330156 (+ 177us) server_negotiation.cc:299] Negotiation successful
0926 16:36:48.330211 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 16:36:48.330452 1863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:48.326774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35422 (local address 127.11.8.252:34881)
0926 16:36:48.326896 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:48.326900 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:48.327039 (+ 139us) server_negotiation.cc:407] Connection header received
0926 16:36:48.327165 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:48.327168 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:48.327208 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:48.327291 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:36:48.327532 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:48.328047 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:48.329255 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:48.329520 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:48.329610 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:48.329952 (+ 342us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:36:48.330027 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:36:48.330069 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:48.330222 (+ 153us) server_negotiation.cc:299] Negotiation successful
0926 16:36:48.330298 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260926 16:36:48.331035 1869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 16:36:48.326021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41456 (local address 127.11.8.254:37221)
0926 16:36:48.326309 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0926 16:36:48.326314 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 16:36:48.326329 (+ 15us) server_negotiation.cc:407] Connection header received
0926 16:36:48.326436 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 16:36:48.326441 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 16:36:48.326612 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 16:36:48.326731 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 16:36:48.327117 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:48.327950 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:48.329148 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 16:36:48.329495 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 16:36:48.329587 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 16:36:48.329985 (+ 398us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 16:36:48.330325 (+ 340us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 16:36:48.330407 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 16:36:48.330841 (+ 434us) server_negotiation.cc:299] Negotiation successful
0926 16:36:48.330899 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":80,"threads_started":1}
I20260926 16:36:48.333986 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 1503
I20260926 16:36:48.340440 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 1578
I20260926 16:36:48.346482 11299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTzWGcB/build/debug/bin/kudu with pid 1654
2026-09-26T16:36:48Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3082 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6177 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (283518 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST